Analysing the use of my shiny apps

This summarises the use of my shiny apps (at https://shiny.psyctc.org/). The analyses will evolve a bit through 2024 as, I hope, the level of use increases.

Current data

Info	Value
First date in data	2024-02-07
Last date in data	2024-11-20
This analysis time/date	03:13 on 21/11/2024
Number of days spanned	287
Total number of sessions	2547
Mean sessions per day	8.87

I am not using any way to separate different users and session is per app, so if someone used multiple apps during one visit to the server, each app used is counts as a separate session.

App uses per day

Here’s the plot of uses per day.

That shows very clearly one large burst of use after the apps were publicised through the Systemic Research Centre Email list (5.iii.24) and a smaller one after a posting to the IDANET list (9.iii.24).

Sessions per week

More sensibly, here is the plot by week, actually plotting the sessions per day and counting from the launch from the launch on 7.ii.24. Where the last week is still an incomplete week that has been taken into account in the calculations. 95% CIs are Poisson model estimates.

Breaking that down by app gives me this.

And facetting by app gives this.

Sessions per Month

This won’t get interesting until the server has been up for quite a lot longer.

The first month was incomplete and the last month will usually be incomplete, that is taken into consideration in computing these session per day rates.

Numbers of sessions per app

Here’s the number of times each app has been used during that period.

App	Sessions	First used	Days available	Sessions per day	Days used	% days used
RCI1	1,048	2024-02-07	288	3.639	201	70%
CSC1	477	2024-02-07	288	1.656	172	60%
Cronbach1Feldt	129	2024-02-07	288	0.448	84	29%
COREpapers1	114	2024-05-11	194	0.588	40	21%
Gaussian1	96	2024-03-05	261	0.368	69	26%
CIcorrelation	89	2024-02-07	288	0.309	51	18%
RCI2	82	2024-02-07	288	0.285	50	17%
CORE-OM_scoring	80	2024-04-16	219	0.365	51	23%
ECDFplot	80	2024-02-07	288	0.278	28	10%
CImean	43	2024-02-07	288	0.149	34	12%
Histogram_and_summary1	38	2024-03-25	241	0.158	19	8%
CIproportion	37	2024-02-07	288	0.128	30	10%
Spearman-Brown	31	2024-05-03	202	0.153	16	8%
plotCIPearson	27	2024-02-07	288	0.094	20	7%
CIdiff2proportions	22	2024-02-07	288	0.076	13	5%
CISpearman	20	2024-02-07	288	0.069	16	6%
Create_univariate_data	20	2024-04-09	226	0.088	19	8%
Screening1	19	2024-02-07	288	0.066	14	5%
g_from_d_and_n	18	2024-02-07	288	0.062	16	6%
CISD	17	2024-02-07	288	0.059	16	6%
CSClookup2a	16	2024-02-07	288	0.056	15	5%
Bonferroni1	15	2024-03-24	242	0.062	12	5%
Attenuation2	12	2024-10-11	41	0.293	9	22%
Attenuation	8	2024-10-09	43	0.186	5	12%
getCorrectedR	8	2024-10-13	39	0.205	4	10%
random1	1	2024-11-19	2	0.500	1	50%

The columns of Sessions per day and of Percentage days used are rather misleading as different apps have been available for very different numbers of days. I won’t be able to get a less misleading forest plot of the mean usage per day per app until there has been far more usage than we have had so far so I will maybe add that later in the year.

However, I can get confidence intervals for proportions on what usage we already have so here’s a less misleading forest plot of proportion of the available days on which each app was used. The dotted reference line marks the overall usage as a proportion of days available across all the apps.

Here’s a map of usage per app against dates. The sizes of the points show how many times the app was used on that day. The y axis sorts by first date used and then by descending total number of times used.

That shows that currently (24.iii.24) most of the apps are shown as first being used on the same day (7.ii.2024) which was the first date I set up the app use logging (and I tested all the then existing apps that day so all appear on the day). The most used apps with come lower on the y axis within first use dates.

Breakdown by day of the week

Weekday	n	percent
Mon	4,032	18%
Tue	4,354	20%
Wed	4,209	19%
Thu	2,678	12%
Fri	3,098	14%
Sat	1,384	6%
Sun	2,250	10%

Same sorted!

Weekday	n	percent
Tue	4,354	20%
Wed	4,209	19%
Mon	4,032	18%
Fri	3,098	14%
Thu	2,678	12%
Sun	2,250	10%
Sat	1,384	6%

Time of day

I’ve broken this down by hour. The server is to some extent protected behind a proxy at my ISP which is good for forcing https access but it does mean that I don’t know where people come from so this is all UMT (i.e. old “GMT”: internet time). I think it also suggests, assuming that most accesses are during working hours, that most visitors/users are coming to the site from Europe or the Americas.

Hour	n	percent
0	16	1%
1	19	1%
2	15	1%
3	11	0%
4	75	3%
5	156	6%
6	167	7%
7	116	5%
8	214	8%
9	175	7%
10	117	5%
11	136	5%
12	166	7%
13	154	6%
14	207	8%
15	132	5%
16	106	4%
17	135	5%
18	91	4%
19	76	3%
20	105	4%
21	96	4%
22	38	1%
23	24	1%

Same sorted.

Hour	n	percent
8	214	8%
14	207	8%
9	175	7%
6	167	7%
12	166	7%
5	156	6%
13	154	6%
11	136	5%
17	135	5%
15	132	5%
10	117	5%
7	116	5%
16	106	4%
20	105	4%
21	96	4%
18	91	4%
19	76	3%
4	75	3%
22	38	1%
23	24	1%
1	19	1%
0	16	1%
2	15	1%
3	11	0%

Browsers

For what little it’s worth, these are the browser IDs picked up by shiny (in descending order of frequency).

Browser	n
Chrome	1,575
Firefox	784
Safari	172
Opera	5

I can’t think it matters but here is the breakdown with the version numbers as well as the browser name.

Browser	n
Chrome 129	256
Chrome 130	212
Chrome 128	191
Chrome 126	138
Firefox 125	132
Chrome 125	121
Firefox 131	120
Chrome 127	116
Chrome 124	111
Safari 17	95
Firefox 130	89
Firefox 129	74
Chrome 122	73
Firefox 132	70
Firefox 124	69
Chrome 101	67
Chrome 123	65
Firefox 128	62
Firefox 123	44
Firefox 126	44
Chrome 86	38
Safari 16	36
Firefox 122	34
Firefox 127	34
Chrome 103	28
Chrome 100	27
Chrome 121	26
Chrome 131	25
Chrome 104	21
Safari 18	19
Chrome 102	17
Chrome 119	16
Safari 604	13
Firefox 119	9
Chrome 120	6
Safari 15	6
Chrome 112	5
Chrome 109	3
Safari 14	3
Chrome 107	2
Chrome 114	2
Chrome 117	2
Firefox 102	2
Opera 109	2
Opera 113	2
Chrome 106	1
Chrome 110	1
Chrome 111	1
Chrome 115	1
Chrome 4	1
Chrome 79	1
Chrome 94	1
Firefox 115	1
Opera 114	1

Durations of sessions

A bit more interesting is the durations of the sessions.
Some sessions don’t have a recorded termination time, currently that’s true for 648, i.e. 25.4% of the sessions. This could include occasional session still active at the time at which the copy of the database was pulled. However, I think most will be where someone leaves the session open. I have capped the sessions at one hour in the analyses below.

Here are the descriptive statistics.

name	nNA	nOK	min	lqrt	mean	uqrt	max
durMinsAll	648	1,899	0.0	1.0	74.1	37.0	1,620.0
durMinsCapped	648	1,899	0.0	1.0	19.8	37.0	60.0
durMinsCensored	1,033	1,514	0.0	1.0	9.5	16.0	60.0

durMinsAll includes all the sessions so far, durMinsCapped treats all sessions recorded as lasting 60 minutes as such, more realistically, durMinsCensored ignores those sessions assuming that they were abandoned sessions. (This shows a maximum duration of 60 minutes as session durations were measured to a fraction of a second so any duration of over 59’30" and less than 60’0" is rounded up to 60 minutes and counted as a genuine 60 minutes!).

Most of the sessions, as you would expect given the nature of the apps, are sessions lasting only a few minutes. If I use the censoring and ignore all the sessions that lasted more than an hour on the plausible assumption that they were abandoned sessions rather than someone continuing to try different parameters for any app for more than an hour then there have been 1514 such sessions so far. Of these 33 lasted under a minute. I guess it’s possible to launch an app and get useful output if only wanting the default parameters in under a minute but I think it would be rare so I think we can regard these as “just looking” sessions and they represent 2.2% of the 1514 uncensored sessions.

The number of sessions lasting a minute (rounding to the nearest minute) was 668, i.e. 44.1% of the uncensored sessions. I think these probably represent very quick but perhaps genuine uses of an app.

That leaves 813 sessions lasting longer than a minute but less than an hour i.e. 53.7% of the uncensored sessions, I think these can be regarded as sessions in which someone entered parameters and perhaps played around with different parameters and perhaps noted or pulled down outputs.

For now (August 2024) I see those as pretty sensible breakdown proportions. I guess that as time goes by it may be interesting to break things down by months and by apps but for now the numbers don’t really merit that and the effects of different apps being added at different times mean that the two variables of app and month are structurally entwined.

Values input

Where it might be useful to me to know more about the usage I am logging input values for some apps. Here’s the breakdown of the numbers of sessions in which inputs were recorded.

app_name	n	percent
COREpapers1	4,861	36.6%
RCI1	4,138	31.1%
CSC1	2,511	18.9%
CImean	419	3.2%
RCI2	385	2.9%
CORE-OM_scoring	161	1.2%
Cronbach1Feldt	154	1.2%
ECDFplot	139	1.0%
Spearman-Brown	136	1.0%
Histogram_and_summary1	135	1.0%
Create_univariate_data	88	0.7%
CIcorrelation	58	0.4%
Gaussian1	27	0.2%
CISpearman	20	0.2%
CIproportion	18	0.1%
random1	12	0.1%
Screening1	7	0.1%
CISD	6	0.0%
CSClookup2a	4	0.0%
plotCIPearson	4	0.0%
g_from_d_and_n	3	0.0%
CIdiff2proportions	2	0.0%

And here are the variables by app, nVisits is the total number of sessions with recorded inputs for that app, nVars is the number of variables that have been input for that app. Finally, nVals is the number of distinct values that have been input for that variable.

app_name	id	nVisits	nVars	nVals
COREpapers1
	authName	4,861	57	102
	date1	4,861	57	45
	date2	4,861	57	31
	embedded	4,861	57	15
	filterAssStructure	4,861	57	16
	filterCORElanguages	4,861	57	15
	filterCOREmeasures	4,861	57	21
	filterFormats	4,861	57	8
	filterGenderCats	4,861	57	7
	mainPlotDownload-filename	4,861	57	3
	mainPlotDownload-format	4,861	57	1
	or	4,861	57	6
	or2	4,861	57	3
	or3	4,861	57	4
	or4	4,861	57	3
	or5	4,861	57	2
	otherMeasure	4,861	57	30
	otherMeasures_cell_clicked	4,861	57	19
	otherMeasures_cells_selected	4,861	57	12
	otherMeasures_columns_selected	4,861	57	12
	otherMeasures_row_last_clicked	4,861	57	5
	otherMeasures_rows_all	4,861	57	61
	otherMeasures_rows_current	4,861	57	60
	otherMeasures_rows_selected	4,861	57	22
	otherMeasures_search	4,861	57	29
	otherMeasures_state	4,861	57	64
	paperLang	4,861	57	11
	papers2_cell_clicked	4,861	57	41
	papers2_cells_selected	4,861	57	16
	papers2_columns_selected	4,861	57	16
	papers2_row_last_clicked	4,861	57	8
	papers2_rows_all	4,861	57	96
	papers2_rows_current	4,861	57	96
	papers2_rows_selected	4,861	57	38
	papers2_search	4,861	57	45
	papers2_state	4,861	57	103
	papers_cell_clicked	4,861	57	137
	papers_cells_selected	4,861	57	108
	papers_columns_selected	4,861	57	108
	papers_row_last_clicked	4,861	57	21
	papers_rows_all	4,861	57	910
	papers_rows_current	4,861	57	927
	papers_rows_selected	4,861	57	177
	papers_search	4,861	57	159
	papers_state	4,861	57	948
	reqEmpCOREdata	4,861	57	24
	reqOA	4,861	57	12
	reqOpenData	4,861	57	5
	reset_input	4,861	57	9
	shinyjs-resettable-side-panel	4,861	57	7
	tabSelected	4,861	57	117
	therOrGen	4,861	57	38
	vecAssStructure	4,861	57	29
	vecCORElanguages	4,861	57	7
	vecFormats	4,861	57	7
	vecGenderCats	4,861	57	7
	vecWhichCOREused	4,861	57	38
RCI1
	SD	4,138	8	1,693
	ci	4,138	8	131
	compute	4,138	8	1,026
	dp	4,138	8	71
	generate	4,138	8	5
	max	4,138	8	2
	min	4,138	8	1
	rel	4,138	8	1,209
CSC1
	SDHS	2,511	7	462
	SDNHS	2,511	7	480
	dp	2,511	7	31
	maxPoss	2,511	7	314
	meanHS	2,511	7	513
	meanNHS	2,511	7	595
	minPoss	2,511	7	116
CImean
	SD	419	5	188
	SE	419	5	1
	dp	419	5	2
	mean	419	5	181
	n	419	5	47
RCI2
	SD	385	6	87
	ci	385	6	23
	compute	385	6	97
	dp	385	6	8
	n	385	6	73
	rel	385	6	97
CORE-OM_scoring
	compData_cell_clicked	161	27	4
	compData_cells_selected	161	27	4
	compData_columns_selected	161	27	4
	compData_rows_all	161	27	4
	compData_rows_current	161	27	10
	compData_rows_selected	161	27	4
	compData_search	161	27	4
	compData_state	161	27	16
	contents_cell_clicked	161	27	2
	contents_cells_selected	161	27	2
	contents_columns_selected	161	27	2
	contents_rows_all	161	27	4
	contents_rows_current	161	27	4
	contents_rows_selected	161	27	2
	contents_search	161	27	2
	contents_state	161	27	4
	dp	161	27	6
	file1	161	27	15
	summary_cell_clicked	161	27	1
	summary_cells_selected	161	27	1
	summary_columns_selected	161	27	1
	summary_rows_all	161	27	1
	summary_rows_current	161	27	1
	summary_rows_selected	161	27	1
	summary_search	161	27	1
	summary_state	161	27	1
	tabSelected	161	27	60
Cronbach1Feldt
	alpha	154	5	87
	ci	154	5	2
	dp	154	5	4
	k	154	5	36
	n	154	5	25
ECDFplot
	annotationSize	139	32	6
	fileHeight	139	32	6
	fileHeightQuantiles	139	32	2
	fileWidth	139	32	6
	fileWidthQuantiles	139	32	2
	inputType	139	32	10
	pastedData	139	32	5
	summary_cell_clicked	139	32	2
	summary_cells_selected	139	32	2
	summary_columns_selected	139	32	2
	summary_rows_all	139	32	4
	summary_rows_current	139	32	4
	summary_rows_selected	139	32	2
	summary_search	139	32	2
	summary_state	139	32	4
	tabSelected	139	32	26
	textSize	139	32	6
	textSizeQuantiles	139	32	2
	tibQuantiles_cell_clicked	139	32	2
	tibQuantiles_cells_selected	139	32	2
	tibQuantiles_columns_selected	139	32	2
	tibQuantiles_rows_all	139	32	4
	tibQuantiles_rows_current	139	32	4
	tibQuantiles_rows_selected	139	32	2
	tibQuantiles_search	139	32	2
	tibQuantiles_state	139	32	4
	title	139	32	6
	titleQuantiles	139	32	2
	xLab	139	32	6
	xLabQuantiles	139	32	2
	yLab	139	32	6
	yLabQuantiles	139	32	2
Spearman-Brown
	currK	136	13	9
	currRel	136	13	11
	maxK	136	13	6
	plotDownload-filename	136	13	1
	reliabilities_cell_clicked	136	13	5
	reliabilities_cells_selected	136	13	5
	reliabilities_columns_selected	136	13	5
	reliabilities_rows_all	136	13	24
	reliabilities_rows_current	136	13	26
	reliabilities_rows_selected	136	13	5
	reliabilities_search	136	13	5
	reliabilities_state	136	13	26
	step	136	13	8
Histogram_and_summary1
	bins	135	24	7
	contents_cell_clicked	135	24	4
	contents_cells_selected	135	24	4
	contents_columns_selected	135	24	4
	contents_rows_all	135	24	8
	contents_rows_current	135	24	8
	contents_rows_selected	135	24	4
	contents_search	135	24	4
	contents_state	135	24	8
	dataType	135	24	6
	file1	135	24	8
	plotDownload-format	135	24	1
	summary_cell_clicked	135	24	3
	summary_cells_selected	135	24	3
	summary_columns_selected	135	24	3
	summary_rows_all	135	24	6
	summary_rows_current	135	24	6
	summary_rows_selected	135	24	3
	summary_search	135	24	3
	summary_state	135	24	6
	title	135	24	8
	var	135	24	9
	xLab	135	24	9
	yLab	135	24	10
Create_univariate_data
	charSeparator	88	11	16
	dataTable_cell_clicked	88	11	5
	dataTable_cells_selected	88	11	5
	dataTable_columns_selected	88	11	5
	dataTable_rows_all	88	11	10
	dataTable_rows_current	88	11	10
	dataTable_rows_selected	88	11	5
	dataTable_search	88	11	5
	dataTable_state	88	11	10
	dist	88	11	2
	generate	88	11	15
CIcorrelation
	R	58	4	31
	ci	58	4	3
	dp	58	4	2
	n	58	4	22
Gaussian1
	dp	27	4	2
	mean	27	4	20
	n	27	4	4
	nBins	27	4	1
CISpearman
	Gaussian	20	5	2
	dp	20	5	1
	method	20	5	6
	n	20	5	3
	rs	20	5	8
CIproportion
	ci	18	4	1
	dp	18	4	2
	n	18	4	6
	x	18	4	9
random1
	compute	12	9	1
	dataTable_cell_clicked	12	9	1
	dataTable_cells_selected	12	9	1
	dataTable_columns_selected	12	9	1
	dataTable_rows_all	12	9	2
	dataTable_rows_current	12	9	2
	dataTable_rows_selected	12	9	1
	dataTable_search	12	9	1
	dataTable_state	12	9	2
Screening1
	prev	7	2	5
	spec	7	2	2
CISD
	SD	6	4	1
	SDorVar	6	4	3
	ci	6	4	1
	n	6	4	1
CSClookup2a
	Lookup	4	1	4
plotCIPearson
	R	4	1	4
g_from_d_and_n
	d	3	2	2
	n	3	2	1
CIdiff2proportions
	n1	2	1	2

So far nVars is a fixed number for each app as it’s going to be maximum number of input values the app requests from the user. Some apps, e.g. RCI1, have a variable “compute” that is just the button instructing the app to run which wasn’t present in early iterations of the app. Another change is that as I get more savvy about shiny some apps, perhaps existing ones, may develop a step-by-step interface so that the numbers of variables input for each use of the app may differ a bit depending on what the user has chosen to do.

Inputs for the RCI1 app

It becomes a bit messy to analyse the inputs as it has to be done (as far as I can currently see) individually by app. It was quite useful as I could see that it had, at least at some point, been possible to enter impossible zero values for reliability and SD. I have now filtered those values out.

Here’s a breakdown for RCI1. These counts only include values that the user entered manually so if the user just left the value at the default value that isn’t counted (however, if the user changes it and then back to the default value, that entry of the default value is counted). I guess I could fix that by filling in the default value where a variable doesn’t appear in the inputs for the session. I’m not sure that’s sufficiently interesting to be worth the faff.

I guess that the .7 entry for the CI was probably me checking the app worked even for that value but I can’t remember for sure. Otherwise it seems entirely sensible that the only other non-default value was .9. The spread of the reliability values is more interesting and looks sensible to me, similarly for the SD.

I guess I could make the app a more interesting information gathering tools if I invited users to input the scale/score being used (i.e. “CORE-OM total”, “BDI-II total”) and even perhaps also ask about dataset (e.g. “my last six months baseline values”, or “the Sheffield X study”) but I think the amount of post-processing that would be necessary to get anything even halfway clean out of that seems unlikely to make this worth the programming/cleaning hassle.

Version history

• Version 9. 20.viii.2024 Tweaks to add histogram of browsers and descriptive statistics for durations of visits.
• Version 8. 2.vii.2024 Corrected mess with rate per day for latest week and first and last months.
• Version 7. 25.vi.2024 Order of sections made more logical, breakdowns by week and month added, polar plot for day in wee.
• Version 6. 8.iv.2024 Fixed mess up arising from an error in Create_univariate_data telemetry.
• Version 5. 1.iv.2024 Tweaks only.
• Version 4. 24.iii.2024 Quite a lot of changes including better aesthetics, time map of use and first analyses of inputs.
• Version 3. 6.iii.2024
  
• Version 2. 16.ii.2024
  
• Version 1. Was probably 14.ii.2024