Statistics on number of messages sent/received via the site per day/week/month/year

[RichardTaylor]

We know we had 100,092 requests made in 2021 https://www.mysociety.org/2021/12/16/whatdotheyknow-transparency-report/

When we compare that to the amount of support mail we deal with, we have a ratio of around one item of "support" mail threads to 13 requests. (Source: Either #1072 or dividing 100,092 by the number of "support" mail messages received in 2021.)

I think that's a really interesting number from the point of view of evaluating how automated the system is. To me it shows what I think is a surprisingly large amount of human intervention required, especially when considering an email thread may contain many messages.

The ratio of support mail threads to messages sent/received via the site would perhaps be another interesting, related, metric.

The average number of messages on a correspondence thread on the site could be derived from statistics on the number of messages sent/received.

[FOIMonkey]

+1 This would be interesting to know.

[garethrees]

One for #1170 if we don't manage it before.

[garethrees]

I think this is probably covered in #1546

[WilliamWDTK]

#1546 only covers the support mailbox, my reading of this issue is that it also covers incoming and outgoing messages on requests in Alaveteli.

[HelenWDTK]

I've had a quick look, and I don't think there's a json feed that I could grab this from to add to the request stats I'm logging. It is shown on the admin stats page, but I'm not aware it is displayed publicly anywhere at the moment.

[garethrees]

Not a feed, but here's a script to generate the data (by month) and a snapshot at the time of posting.

class MessageCountsPerMonth
  def to_csv
    CSV.generate do |csv|
      csv << message_counts.first.keys

      message_counts.each do |month|
        csv << month.values
      end
    end
  end

  def message_counts
    incoming_counts = IncomingMessage.group("DATE_TRUNC('month', created_at)").count
    outgoing_counts = OutgoingMessage.group("DATE_TRUNC('month', created_at)").count

    all_months = (incoming_counts.keys + outgoing_counts.keys).uniq.sort

    result = all_months.map do |month|
      {
        month: month,
        incoming_count: incoming_counts[month] || 0,
        outgoing_count: outgoing_counts[month] || 0
      }
    end

    result
  end
end

File.write('tmp/message-counts.csv', MessageCountsPerMonth.new.to_csv)

message_counts_per_month.csv

month	incoming_count	outgoing_count
2007-12-01 00:00:00 UTC	8	5
2008-01-01 00:00:00 UTC	22	13
2008-02-01 00:00:00 UTC	40	34
2008-03-01 00:00:00 UTC	97	104
2008-04-01 00:00:00 UTC	287	242
2008-05-01 00:00:00 UTC	461	438
2008-06-01 00:00:00 UTC	677	486
2008-07-01 00:00:00 UTC	1061	848
2008-08-01 00:00:00 UTC	1839	1639
2008-09-01 00:00:00 UTC	2016	1253
2008-10-01 00:00:00 UTC	1503	1075
2008-11-01 00:00:00 UTC	1339	994
2008-12-01 00:00:00 UTC	1703	1633
2009-01-01 00:00:00 UTC	3619	3621
2009-02-01 00:00:00 UTC	3623	2627
2009-03-01 00:00:00 UTC	3620	2676
2009-04-01 00:00:00 UTC	3061	2483
2009-05-01 00:00:00 UTC	3294	2915
2009-06-01 00:00:00 UTC	3864	2847
2009-07-01 00:00:00 UTC	3908	3156
2009-08-01 00:00:00 UTC	4755	3446
2009-09-01 00:00:00 UTC	4027	3308
2009-10-01 00:00:00 UTC	6669	4937
2009-11-01 00:00:00 UTC	4984	3582
2009-12-01 00:00:00 UTC	4041	2842
2010-01-01 00:00:00 UTC	6010	4532
2010-02-01 00:00:00 UTC	4628	3484
2010-03-01 00:00:00 UTC	5222	3819
2010-04-01 00:00:00 UTC	5720	4282
2010-05-01 00:00:00 UTC	4746	3305
2010-06-01 00:00:00 UTC	7401	6084
2010-07-01 00:00:00 UTC	7649	5235
2010-08-01 00:00:00 UTC	7069	4855
2010-09-01 00:00:00 UTC	6320	4304
2010-10-01 00:00:00 UTC	5492	4150
2010-11-01 00:00:00 UTC	6269	4421
2010-12-01 00:00:00 UTC	6650	4596
2011-01-01 00:00:00 UTC	9671	7593
2011-02-01 00:00:00 UTC	8648	5740
2011-03-01 00:00:00 UTC	8567	6202
2011-04-01 00:00:00 UTC	6672	5132
2011-05-01 00:00:00 UTC	7864	5827
2011-06-01 00:00:00 UTC	10676	7730
2011-07-01 00:00:00 UTC	8190	5193
2011-08-01 00:00:00 UTC	8328	5717
2011-09-01 00:00:00 UTC	8170	5692
2011-10-01 00:00:00 UTC	8560	6273
2011-11-01 00:00:00 UTC	8379	6072
2011-12-01 00:00:00 UTC	7433	5147
2012-01-01 00:00:00 UTC	10745	8045
2012-02-01 00:00:00 UTC	9789	5991
2012-03-01 00:00:00 UTC	9573	7086
2012-04-01 00:00:00 UTC	8100	5371
2012-05-01 00:00:00 UTC	8278	5265
2012-06-01 00:00:00 UTC	6133	4289
2012-07-01 00:00:00 UTC	7276	5048
2012-08-01 00:00:00 UTC	8728	6266
2012-09-01 00:00:00 UTC	7709	5133
2012-10-01 00:00:00 UTC	10201	7082
2012-11-01 00:00:00 UTC	10637	6844
2012-12-01 00:00:00 UTC	7769	4959
2013-01-01 00:00:00 UTC	10237	7159
2013-02-01 00:00:00 UTC	9320	6141
2013-03-01 00:00:00 UTC	8901	6122
2013-04-01 00:00:00 UTC	10884	7003
2013-05-01 00:00:00 UTC	9905	6145
2013-06-01 00:00:00 UTC	8882	6148
2013-07-01 00:00:00 UTC	10428	6444
2013-08-01 00:00:00 UTC	10185	6342
2013-09-01 00:00:00 UTC	9967	6824
2013-10-01 00:00:00 UTC	10198	6405
2013-11-01 00:00:00 UTC	10215	7421
2013-12-01 00:00:00 UTC	9893	6403
2014-01-01 00:00:00 UTC	11040	7153
2014-02-01 00:00:00 UTC	10030	6874
2014-03-01 00:00:00 UTC	11216	7658
2014-04-01 00:00:00 UTC	11464	7433
2014-05-01 00:00:00 UTC	12233	8521
2014-06-01 00:00:00 UTC	10694	6542
2014-07-01 00:00:00 UTC	10806	6953
2014-08-01 00:00:00 UTC	10209	7337
2014-09-01 00:00:00 UTC	11071	7148
2014-10-01 00:00:00 UTC	11646	7462
2014-11-01 00:00:00 UTC	10812	7583
2014-12-01 00:00:00 UTC	10403	6426
2015-01-01 00:00:00 UTC	11398	7840
2015-02-01 00:00:00 UTC	10544	6866
2015-03-01 00:00:00 UTC	12251	8322
2015-04-01 00:00:00 UTC	11561	7229
2015-05-01 00:00:00 UTC	10606	7224
2015-06-01 00:00:00 UTC	14290	9103
2015-07-01 00:00:00 UTC	14188	8641
2015-08-01 00:00:00 UTC	12013	8020
2015-09-01 00:00:00 UTC	13528	8533
2015-10-01 00:00:00 UTC	12782	8328
2015-11-01 00:00:00 UTC	12541	7826
2015-12-01 00:00:00 UTC	9741	5562
2016-01-01 00:00:00 UTC	10681	7358
2016-02-01 00:00:00 UTC	13466	8662
2016-03-01 00:00:00 UTC	15106	9667
2016-04-01 00:00:00 UTC	15116	9490
2016-05-01 00:00:00 UTC	13488	8609
2016-06-01 00:00:00 UTC	12895	8522
2016-07-01 00:00:00 UTC	13802	9246
2016-08-01 00:00:00 UTC	14600	9313
2016-09-01 00:00:00 UTC	14535	9336
2016-10-01 00:00:00 UTC	12728	8195
2016-11-01 00:00:00 UTC	13809	9059
2016-12-01 00:00:00 UTC	11645	7258
2017-01-01 00:00:00 UTC	14223	9927
2017-02-01 00:00:00 UTC	14900	9259
2017-03-01 00:00:00 UTC	14669	9215
2017-04-01 00:00:00 UTC	11912	8009
2017-05-01 00:00:00 UTC	13326	8297
2017-06-01 00:00:00 UTC	13760	8686
2017-07-01 00:00:00 UTC	14909	10338
2017-08-01 00:00:00 UTC	15351	9607
2017-09-01 00:00:00 UTC	15192	10020
2017-10-01 00:00:00 UTC	16196	10608
2017-11-01 00:00:00 UTC	16309	10820
2017-12-01 00:00:00 UTC	11498	7710
2018-01-01 00:00:00 UTC	15048	10451
2018-02-01 00:00:00 UTC	14898	9546
2018-03-01 00:00:00 UTC	15081	10058
2018-04-01 00:00:00 UTC	15582	9944
2018-05-01 00:00:00 UTC	15822	10235
2018-06-01 00:00:00 UTC	14940	10125
2018-07-01 00:00:00 UTC	18454	13252
2018-08-01 00:00:00 UTC	24337	16542
2018-09-01 00:00:00 UTC	16399	9262
2018-10-01 00:00:00 UTC	16540	10427
2018-11-01 00:00:00 UTC	15544	10111
2018-12-01 00:00:00 UTC	12366	7412
2019-01-01 00:00:00 UTC	16897	10984
2019-02-01 00:00:00 UTC	17464	11397
2019-03-01 00:00:00 UTC	18511	11591
2019-04-01 00:00:00 UTC	17053	10572
2019-05-01 00:00:00 UTC	18734	11287
2019-06-01 00:00:00 UTC	14497	8603
2019-07-01 00:00:00 UTC	18065	11591
2019-08-01 00:00:00 UTC	16189	9705
2019-09-01 00:00:00 UTC	16455	10570
2019-10-01 00:00:00 UTC	19566	11779
2019-11-01 00:00:00 UTC	16984	11018
2019-12-01 00:00:00 UTC	15358	9194
2020-01-01 00:00:00 UTC	20689	13356
2020-02-01 00:00:00 UTC	23338	14744
2020-03-01 00:00:00 UTC	17550	8657
2020-04-01 00:00:00 UTC	10860	6206
2020-05-01 00:00:00 UTC	13101	8148
2020-06-01 00:00:00 UTC	14310	8123
2020-07-01 00:00:00 UTC	16880	10272
2020-08-01 00:00:00 UTC	19894	10432
2020-09-01 00:00:00 UTC	18133	10417
2020-10-01 00:00:00 UTC	18535	10706
2020-11-01 00:00:00 UTC	16772	10067
2020-12-01 00:00:00 UTC	13400	7357
2021-01-01 00:00:00 UTC	16869	10706
2021-02-01 00:00:00 UTC	19624	12288
2021-03-01 00:00:00 UTC	24485	15072
2021-04-01 00:00:00 UTC	21997	12762
2021-05-01 00:00:00 UTC	20685	13085
2021-06-01 00:00:00 UTC	21926	12516
2021-07-01 00:00:00 UTC	21480	12371
2021-08-01 00:00:00 UTC	19334	11871
2021-09-01 00:00:00 UTC	22663	12797
2021-10-01 00:00:00 UTC	17885	10232
2021-11-01 00:00:00 UTC	20503	12825
2021-12-01 00:00:00 UTC	17019	8615
2022-01-01 00:00:00 UTC	20430	13444
2022-02-01 00:00:00 UTC	20255	11931
2022-03-01 00:00:00 UTC	25786	16271
2022-04-01 00:00:00 UTC	20497	12077
2022-05-01 00:00:00 UTC	20196	12181
2022-06-01 00:00:00 UTC	20068	11685
2022-07-01 00:00:00 UTC	20234	12240
2022-08-01 00:00:00 UTC	21055	13038
2022-09-01 00:00:00 UTC	24162	14391
2022-10-01 00:00:00 UTC	19742	12045
2022-11-01 00:00:00 UTC	23183	15903
2022-12-01 00:00:00 UTC	17943	9050
2023-01-01 00:00:00 UTC	24662	15743
2023-02-01 00:00:00 UTC	25950	14698
2023-03-01 00:00:00 UTC	28632	16123
2023-04-01 00:00:00 UTC	14404	8064

[garethrees]

Don't think we need to automate this right now so closing. Perhaps one to think about nearer #1536.

[garethrees]

[garethrees]

We know we had 100,092 requests made in 2021 https://www.mysociety.org/2021/12/16/whatdotheyknow-transparency-report/

When we compare that to the amount of support mail we deal with, we have a ratio of around one item of "support" mail threads to 13 requests. (Source: Either #1072 or dividing 100,092 by the number of "support" mail messages received in 2021.)

Just noting here that using the same formula with the 2022 report we have 109,653 requests / 8,912 inbox threads = 12.3

[confirmordeny]

That's a positive change.