Analysing the use of my shiny apps

This summarises the use of my shiny apps (at https://shiny.psyctc.org/). The analyses will evolve a bit through 2024 as, I hope, the level of use increases.

Current data

Info	Value
First date in data	2024-02-07
Last date in data	2024-12-20
This analysis time/date	03:13 on 21/12/2024
Number of days spanned	317
Total number of sessions	2963
Mean sessions per day	9.35

I am not using any way to separate different users and session is per app, so if someone used multiple apps during one visit to the server, each app used is counts as a separate session.

App uses per day

Here’s the plot of uses per day.

That shows very clearly one large burst of use after the apps were publicised through the Systemic Research Centre Email list (5.iii.24) and a smaller one after a posting to the IDANET list (9.iii.24).

Sessions per week

More sensibly, here is the plot by week, actually plotting the sessions per day and counting from the launch from the launch on 7.ii.24. Where the last week is still an incomplete week that has been taken into account in the calculations. 95% CIs are Poisson model estimates.

Breaking that down by app gives me this.

And facetting by app gives this.

Sessions per Month

This won’t get interesting until the server has been up for quite a lot longer.

The first month was incomplete and the last month will usually be incomplete, that is taken into consideration in computing these session per day rates.

Numbers of sessions per app

Here’s the number of times each app has been used during that period.

App	Sessions	First used	Days available	Sessions per day	Days used	% days used
RCI1	1,205	2024-02-07	318	3.789	231	73%
CSC1	527	2024-02-07	318	1.657	190	60%
COREpapers1	166	2024-05-11	224	0.741	69	31%
Cronbach1Feldt	143	2024-02-07	318	0.450	91	29%
RCI2	115	2024-02-07	318	0.362	64	20%
CORE-OM_scoring	100	2024-04-16	249	0.402	60	24%
Gaussian1	99	2024-03-05	291	0.340	72	25%
CIcorrelation	94	2024-02-07	318	0.296	55	17%
ECDFplot	82	2024-02-07	318	0.258	30	9%
CImean	44	2024-02-07	318	0.138	35	11%
CSClookup2a	43	2024-02-07	318	0.135	23	7%
CIproportion	41	2024-02-07	318	0.129	33	10%
Histogram_and_summary1	38	2024-03-25	271	0.140	19	7%
Spearman-Brown	34	2024-05-03	232	0.147	19	8%
plotCIPearson	29	2024-02-07	318	0.091	22	7%
CISpearman	26	2024-02-07	318	0.082	22	7%
CIdiff2proportions	23	2024-02-07	318	0.072	14	4%
Create_univariate_data	20	2024-04-09	256	0.078	19	7%
Screening1	20	2024-02-07	318	0.063	15	5%
g_from_d_and_n	19	2024-02-07	318	0.060	17	5%
CISD	18	2024-02-07	318	0.057	17	5%
Bonferroni1	17	2024-03-24	272	0.062	14	5%
Attenuation2	15	2024-10-11	71	0.211	12	17%
Attenuation	13	2024-10-09	73	0.178	10	14%
Feldt2	13	2024-11-27	24	0.542	10	42%
random1	10	2024-11-19	32	0.312	9	28%
getCorrectedR	9	2024-10-13	69	0.130	5	7%

The columns of Sessions per day and of Percentage days used are rather misleading as different apps have been available for very different numbers of days. I won’t be able to get a less misleading forest plot of the mean usage per day per app until there has been far more usage than we have had so far so I will maybe add that later in the year.

However, I can get confidence intervals for proportions on what usage we already have so here’s a less misleading forest plot of proportion of the available days on which each app was used. The dotted reference line marks the overall usage as a proportion of days available across all the apps.

Here’s a map of usage per app against dates. The sizes of the points show how many times the app was used on that day. The y axis sorts by first date used and then by descending total number of times used.

That shows that currently (24.iii.24) most of the apps are shown as first being used on the same day (7.ii.2024) which was the first date I set up the app use logging (and I tested all the then existing apps that day so all appear on the day). The most used apps with come lower on the y axis within first use dates.

Breakdown by day of the week

Weekday	n	percent
Mon	4,593	18%
Tue	4,983	19%
Wed	4,607	18%
Thu	3,730	14%
Fri	3,771	14%
Sat	1,760	7%
Sun	2,577	10%

Same sorted!

Weekday	n	percent
Tue	4,983	19%
Wed	4,607	18%
Mon	4,593	18%
Fri	3,771	14%
Thu	3,730	14%
Sun	2,577	10%
Sat	1,760	7%

Time of day

I’ve broken this down by hour. The server is to some extent protected behind a proxy at my ISP which is good for forcing https access but it does mean that I don’t know where people come from so this is all UMT (i.e. old “GMT”: internet time). I think it also suggests, assuming that most accesses are during working hours, that most visitors/users are coming to the site from Europe or the Americas.

Hour	n	percent
0	24	1%
1	26	1%
2	21	1%
3	22	1%
4	93	3%
5	160	5%
6	181	6%
7	140	5%
8	246	8%
9	199	7%
10	150	5%
11	160	5%
12	186	6%
13	168	6%
14	225	8%
15	154	5%
16	121	4%
17	158	5%
18	109	4%
19	94	3%
20	133	4%
21	107	4%
22	50	2%
23	36	1%

Same sorted.

Hour	n	percent
8	246	8%
14	225	8%
9	199	7%
12	186	6%
6	181	6%
13	168	6%
5	160	5%
11	160	5%
17	158	5%
15	154	5%
10	150	5%
7	140	5%
20	133	4%
16	121	4%
18	109	4%
21	107	4%
19	94	3%
4	93	3%
22	50	2%
23	36	1%
1	26	1%
0	24	1%
3	22	1%
2	21	1%

Browsers

For what little it’s worth, these are the browser IDs picked up by shiny (in descending order of frequency).

Browser	n
Chrome	1,830
Firefox	892
Safari	212
Netscape.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/) -?	10
Opera	5

I can’t think it matters but here is the breakdown with the version numbers as well as the browser name.

Browser	n
Chrome 129	263
Chrome 130	243
Chrome 131	196
Chrome 128	194
Chrome 125	150
Chrome 126	140
Firefox 125	132
Chrome 127	120
Firefox 131	120
Chrome 124	111
Safari 17	107
Firefox 133	90
Firefox 130	89
Firefox 132	85
Chrome 122	75
Firefox 129	74
Firefox 124	69
Chrome 101	67
Chrome 123	65
Firefox 128	63
Firefox 123	44
Firefox 126	44
Safari 18	42
Chrome 86	38
Safari 16	36
Firefox 122	34
Firefox 127	34
Chrome 103	28
Chrome 100	27
Chrome 121	26
Chrome 104	21
Chrome 102	17
Chrome 119	16
Safari 604	16
Netscape 5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/) -?	10
Firefox 119	9
Safari 15	8
Chrome 120	6
Chrome 112	5
Chrome 106	4
Chrome 109	3
Firefox 115	3
Safari 14	3
Chrome 107	2
Chrome 114	2
Chrome 116	2
Chrome 117	2
Chrome 79	2
Firefox 102	2
Opera 109	2
Opera 113	2
Chrome 110	1
Chrome 111	1
Chrome 115	1
Chrome 4	1
Chrome 94	1
Opera 114	1

Durations of sessions

A bit more interesting is the durations of the sessions.
Some sessions don’t have a recorded termination time, currently that’s true for 709, i.e. 23.9% of the sessions. This could include occasional session still active at the time at which the copy of the database was pulled. However, I think most will be where someone leaves the session open. I have capped the sessions at one hour in the analyses below.

Here are the descriptive statistics.

name	nNA	nOK	min	lqrt	mean	uqrt	max
durMinsAll	709	2,254	0.0	1.0	75.3	39.8	1,620.0
durMinsCapped	709	2,254	0.0	1.0	20.0	39.8	60.0
durMinsCensored	1,178	1,785	0.0	1.0	9.5	16.0	60.0

durMinsAll includes all the sessions so far, durMinsCapped treats all sessions recorded as lasting 60 minutes as such, more realistically, durMinsCensored ignores those sessions assuming that they were abandoned sessions. (This shows a maximum duration of 60 minutes as session durations were measured to a fraction of a second so any duration of over 59’30" and less than 60’0" is rounded up to 60 minutes and counted as a genuine 60 minutes!).

Most of the sessions, as you would expect given the nature of the apps, are sessions lasting only a few minutes. If I use the censoring and ignore all the sessions that lasted more than an hour on the plausible assumption that they were abandoned sessions rather than someone continuing to try different parameters for any app for more than an hour then there have been 1785 such sessions so far. Of these 37 lasted under a minute. I guess it’s possible to launch an app and get useful output if only wanting the default parameters in under a minute but I think it would be rare so I think we can regard these as “just looking” sessions and they represent 2.1% of the 1785 uncensored sessions.

The number of sessions lasting a minute (rounding to the nearest minute) was 810, i.e. 45.4% of the uncensored sessions. I think these probably represent very quick but perhaps genuine uses of an app.

That leaves 938 sessions lasting longer than a minute but less than an hour i.e. 52.5% of the uncensored sessions, I think these can be regarded as sessions in which someone entered parameters and perhaps played around with different parameters and perhaps noted or pulled down outputs.

For now (August 2024) I see those as pretty sensible breakdown proportions. I guess that as time goes by it may be interesting to break things down by months and by apps but for now the numbers don’t really merit that and the effects of different apps being added at different times mean that the two variables of app and month are structurally entwined.

Values input

Where it might be useful to me to know more about the usage I am logging input values for some apps. Here’s the breakdown of the numbers of sessions in which inputs were recorded.

app_name	n	percent
COREpapers1	6,056	38.1%
RCI1	4,970	31.3%
CSC1	2,705	17.0%
RCI2	466	2.9%
CImean	419	2.6%
CORE-OM_scoring	177	1.1%
Cronbach1Feldt	168	1.1%
ECDFplot	139	0.9%
Spearman-Brown	136	0.9%
CSClookup2a	135	0.8%
Histogram_and_summary1	135	0.8%
random1	122	0.8%
Create_univariate_data	88	0.6%
CIcorrelation	72	0.5%
CISpearman	28	0.2%
Gaussian1	27	0.2%
CIproportion	18	0.1%
Feldt2	12	0.1%
Screening1	7	0.0%
CISD	6	0.0%
plotCIPearson	4	0.0%
g_from_d_and_n	3	0.0%
CIdiff2proportions	2	0.0%

And here are the variables by app, nVisits is the total number of sessions with recorded inputs for that app, nVars is the number of variables that have been input for that app. Finally, nVals is the number of distinct values that have been input for that variable.

app_name	id	nVisits	nVars	nVals
COREpapers1
	authName	6,056	58	102
	clipbtn	6,056	58	1
	date1	6,056	58	45
	date2	6,056	58	31
	embedded	6,056	58	17
	filterAssStructure	6,056	58	18
	filterCORElanguages	6,056	58	15
	filterCOREmeasures	6,056	58	25
	filterFormats	6,056	58	17
	filterGenderCats	6,056	58	9
	mainPlotDownload-filename	6,056	58	3
	mainPlotDownload-format	6,056	58	1
	or	6,056	58	6
	or2	6,056	58	3
	or3	6,056	58	4
	or4	6,056	58	3
	or5	6,056	58	4
	otherMeasure	6,056	58	30
	otherMeasures_cell_clicked	6,056	58	19
	otherMeasures_cells_selected	6,056	58	12
	otherMeasures_columns_selected	6,056	58	12
	otherMeasures_row_last_clicked	6,056	58	5
	otherMeasures_rows_all	6,056	58	61
	otherMeasures_rows_current	6,056	58	60
	otherMeasures_rows_selected	6,056	58	22
	otherMeasures_search	6,056	58	29
	otherMeasures_state	6,056	58	64
	paperLang	6,056	58	23
	papers2_cell_clicked	6,056	58	41
	papers2_cells_selected	6,056	58	16
	papers2_columns_selected	6,056	58	16
	papers2_row_last_clicked	6,056	58	8
	papers2_rows_all	6,056	58	96
	papers2_rows_current	6,056	58	96
	papers2_rows_selected	6,056	58	38
	papers2_search	6,056	58	45
	papers2_state	6,056	58	103
	papers_cell_clicked	6,056	58	197
	papers_cells_selected	6,056	58	158
	papers_columns_selected	6,056	58	158
	papers_row_last_clicked	6,056	58	30
	papers_rows_all	6,056	58	1,180
	papers_rows_current	6,056	58	1,205
	papers_rows_selected	6,056	58	242
	papers_search	6,056	58	209
	papers_state	6,056	58	1,226
	reqEmpCOREdata	6,056	58	34
	reqOA	6,056	58	12
	reqOpenData	6,056	58	13
	reset_input	6,056	58	9
	shinyjs-resettable-side-panel	6,056	58	7
	tabSelected	6,056	58	117
	therOrGen	6,056	58	45
	vecAssStructure	6,056	58	29
	vecCORElanguages	6,056	58	7
	vecFormats	6,056	58	19
	vecGenderCats	6,056	58	8
	vecWhichCOREused	6,056	58	51
RCI1
	SD	4,970	8	1,972
	ci	4,970	8	207
	compute	4,970	8	1,253
	dp	4,970	8	76
	generate	4,970	8	5
	max	4,970	8	2
	min	4,970	8	1
	rel	4,970	8	1,454
CSC1
	SDHS	2,705	7	495
	SDNHS	2,705	7	510
	dp	2,705	7	31
	maxPoss	2,705	7	370
	meanHS	2,705	7	541
	meanNHS	2,705	7	629
	minPoss	2,705	7	129
RCI2
	SD	466	6	109
	ci	466	6	23
	compute	466	6	124
	dp	466	6	8
	n	466	6	89
	rel	466	6	113
CImean
	SD	419	5	188
	SE	419	5	1
	dp	419	5	2
	mean	419	5	181
	n	419	5	47
CORE-OM_scoring
	compData_cell_clicked	177	27	4
	compData_cells_selected	177	27	4
	compData_columns_selected	177	27	4
	compData_rows_all	177	27	4
	compData_rows_current	177	27	10
	compData_rows_selected	177	27	4
	compData_search	177	27	4
	compData_state	177	27	16
	contents_cell_clicked	177	27	2
	contents_cells_selected	177	27	2
	contents_columns_selected	177	27	2
	contents_rows_all	177	27	4
	contents_rows_current	177	27	4
	contents_rows_selected	177	27	2
	contents_search	177	27	2
	contents_state	177	27	4
	dp	177	27	7
	file1	177	27	15
	summary_cell_clicked	177	27	1
	summary_cells_selected	177	27	1
	summary_columns_selected	177	27	1
	summary_rows_all	177	27	1
	summary_rows_current	177	27	1
	summary_rows_selected	177	27	1
	summary_search	177	27	1
	summary_state	177	27	1
	tabSelected	177	27	75
Cronbach1Feldt
	alpha	168	5	91
	ci	168	5	2
	dp	168	5	4
	k	168	5	38
	n	168	5	33
ECDFplot
	annotationSize	139	32	6
	fileHeight	139	32	6
	fileHeightQuantiles	139	32	2
	fileWidth	139	32	6
	fileWidthQuantiles	139	32	2
	inputType	139	32	10
	pastedData	139	32	5
	summary_cell_clicked	139	32	2
	summary_cells_selected	139	32	2
	summary_columns_selected	139	32	2
	summary_rows_all	139	32	4
	summary_rows_current	139	32	4
	summary_rows_selected	139	32	2
	summary_search	139	32	2
	summary_state	139	32	4
	tabSelected	139	32	26
	textSize	139	32	6
	textSizeQuantiles	139	32	2
	tibQuantiles_cell_clicked	139	32	2
	tibQuantiles_cells_selected	139	32	2
	tibQuantiles_columns_selected	139	32	2
	tibQuantiles_rows_all	139	32	4
	tibQuantiles_rows_current	139	32	4
	tibQuantiles_rows_selected	139	32	2
	tibQuantiles_search	139	32	2
	tibQuantiles_state	139	32	4
	title	139	32	6
	titleQuantiles	139	32	2
	xLab	139	32	6
	xLabQuantiles	139	32	2
	yLab	139	32	6
	yLabQuantiles	139	32	2
Spearman-Brown
	currK	136	13	9
	currRel	136	13	11
	maxK	136	13	6
	plotDownload-filename	136	13	1
	reliabilities_cell_clicked	136	13	5
	reliabilities_cells_selected	136	13	5
	reliabilities_columns_selected	136	13	5
	reliabilities_rows_all	136	13	24
	reliabilities_rows_current	136	13	26
	reliabilities_rows_selected	136	13	5
	reliabilities_search	136	13	5
	reliabilities_state	136	13	26
	step	136	13	8
CSClookup2a
	Age	135	5	2
	Gender	135	5	1
	Lookup	135	5	13
	Scoring	135	5	13
	YPscore	135	5	106
Histogram_and_summary1
	bins	135	24	7
	contents_cell_clicked	135	24	4
	contents_cells_selected	135	24	4
	contents_columns_selected	135	24	4
	contents_rows_all	135	24	8
	contents_rows_current	135	24	8
	contents_rows_selected	135	24	4
	contents_search	135	24	4
	contents_state	135	24	8
	dataType	135	24	6
	file1	135	24	8
	plotDownload-format	135	24	1
	summary_cell_clicked	135	24	3
	summary_cells_selected	135	24	3
	summary_columns_selected	135	24	3
	summary_rows_all	135	24	6
	summary_rows_current	135	24	6
	summary_rows_selected	135	24	3
	summary_search	135	24	3
	summary_state	135	24	6
	title	135	24	8
	var	135	24	9
	xLab	135	24	9
	yLab	135	24	10
random1
	compute	122	11	10
	dataTable_cell_clicked	122	11	9
	dataTable_cells_selected	122	11	9
	dataTable_columns_selected	122	11	9
	dataTable_rows_all	122	11	20
	dataTable_rows_current	122	11	20
	dataTable_rows_selected	122	11	9
	dataTable_search	122	11	9
	dataTable_state	122	11	20
	valN	122	11	6
	valSeed	122	11	1
Create_univariate_data
	charSeparator	88	11	16
	dataTable_cell_clicked	88	11	5
	dataTable_cells_selected	88	11	5
	dataTable_columns_selected	88	11	5
	dataTable_rows_all	88	11	10
	dataTable_rows_current	88	11	10
	dataTable_rows_selected	88	11	5
	dataTable_search	88	11	5
	dataTable_state	88	11	10
	dist	88	11	2
	generate	88	11	15
CIcorrelation
	R	72	4	37
	ci	72	4	3
	dp	72	4	2
	n	72	4	30
CISpearman
	Gaussian	28	5	2
	dp	28	5	1
	method	28	5	6
	n	28	5	11
	rs	28	5	8
Gaussian1
	dp	27	4	2
	mean	27	4	20
	n	27	4	4
	nBins	27	4	1
CIproportion
	ci	18	4	1
	dp	18	4	2
	n	18	4	6
	x	18	4	9
Feldt2
	alpha1	12	5	1
	alpha2	12	5	3
	dp	12	5	3
	n1	12	5	3
	n2	12	5	2
Screening1
	prev	7	2	5
	spec	7	2	2
CISD
	SD	6	4	1
	SDorVar	6	4	3
	ci	6	4	1
	n	6	4	1
plotCIPearson
	R	4	1	4
g_from_d_and_n
	d	3	2	2
	n	3	2	1
CIdiff2proportions
	n1	2	1	2

So far nVars is a fixed number for each app as it’s going to be maximum number of input values the app requests from the user. Some apps, e.g. RCI1, have a variable “compute” that is just the button instructing the app to run which wasn’t present in early iterations of the app. Another change is that as I get more savvy about shiny some apps, perhaps existing ones, may develop a step-by-step interface so that the numbers of variables input for each use of the app may differ a bit depending on what the user has chosen to do.

Inputs for the RCI1 app

It becomes a bit messy to analyse the inputs as it has to be done (as far as I can currently see) individually by app. It was quite useful as I could see that it had, at least at some point, been possible to enter impossible zero values for reliability and SD. I have now filtered those values out.

Here’s a breakdown for RCI1. These counts only include values that the user entered manually so if the user just left the value at the default value that isn’t counted (however, if the user changes it and then back to the default value, that entry of the default value is counted). I guess I could fix that by filling in the default value where a variable doesn’t appear in the inputs for the session. I’m not sure that’s sufficiently interesting to be worth the faff.

I guess that the .7 entry for the CI was probably me checking the app worked even for that value but I can’t remember for sure. Otherwise it seems entirely sensible that the only other non-default value was .9. The spread of the reliability values is more interesting and looks sensible to me, similarly for the SD.

I guess I could make the app a more interesting information gathering tools if I invited users to input the scale/score being used (i.e. “CORE-OM total”, “BDI-II total”) and even perhaps also ask about dataset (e.g. “my last six months baseline values”, or “the Sheffield X study”) but I think the amount of post-processing that would be necessary to get anything even halfway clean out of that seems unlikely to make this worth the programming/cleaning hassle.

Version history

• Version 9. 20.viii.2024 Tweaks to add histogram of browsers and descriptive statistics for durations of visits.
• Version 8. 2.vii.2024 Corrected mess with rate per day for latest week and first and last months.
• Version 7. 25.vi.2024 Order of sections made more logical, breakdowns by week and month added, polar plot for day in wee.
• Version 6. 8.iv.2024 Fixed mess up arising from an error in Create_univariate_data telemetry.
• Version 5. 1.iv.2024 Tweaks only.
• Version 4. 24.iii.2024 Quite a lot of changes including better aesthetics, time map of use and first analyses of inputs.
• Version 3. 6.iii.2024
  
• Version 2. 16.ii.2024
  
• Version 1. Was probably 14.ii.2024