Replies: 2 comments
-
@jqb101 Major, I know that you have asked the same question on Twitter Developer Community and the developer of twarc2 has given you some advice. The developer of twarc tried it only with the count endpoint. Here I tried both the count and the search end points. What I want to tell you is that it is not likely to be an issue with the package. It can indeed get the data from the time range you mentioned. Instead, the Twitter search index is very wonky with old tweets. My suggestion is not to collect your tweets in one query but to break it down into multiple queries and explore what went wrong. require(academictwitteR)
#> Loading required package: academictwitteR
wapo_counts <- count_all_tweets(users = "washingtonpost", start_tweets = "2009-08-01T00:00:00Z", end_tweets = "2009-10-31T00:00:00Z", is_retweet=FALSE, is_reply=FALSE, n = 300)
#> query: (from:washingtonpost) -is:retweet -is:reply
#> Total pages queried: 1 (tweets captured this page: 31).
#> Total pages queried: 2 (tweets captured this page: 31).
#> Total pages queried: 3 (tweets captured this page: 29).
#> This is the last page for (from:washingtonpost) -is:retweet -is:reply : finishing collection.
sum(wapo_counts$tweet_count)
#> [1] 956
wapo_ft <- get_all_tweets(users = "washingtonpost", start_tweets = "2009-08-01T00:00:00Z", end_tweets = "2009-10-31T00:00:00Z", is_retweet=FALSE, is_reply=FALSE, n = 1000, verbose = FALSE)
wapo_ft$created_at
#> [1] "2009-10-30T23:05:46.000Z" "2009-10-30T21:16:30.000Z"
#> [3] "2009-10-30T20:43:13.000Z" "2009-10-30T20:04:56.000Z"
#> [5] "2009-10-30T19:35:23.000Z" "2009-10-30T18:28:17.000Z"
#> [7] "2009-10-30T18:08:17.000Z" "2009-10-30T18:03:19.000Z"
#> [9] "2009-10-30T17:08:14.000Z" "2009-10-30T15:53:29.000Z"
#> [11] "2009-10-30T15:47:15.000Z" "2009-10-30T14:51:28.000Z"
#> [13] "2009-10-30T13:59:50.000Z" "2009-10-30T13:51:52.000Z"
#> [15] "2009-10-30T13:25:33.000Z" "2009-10-30T13:04:56.000Z"
#> [17] "2009-10-30T12:44:31.000Z" "2009-10-30T12:43:52.000Z"
#> [19] "2009-10-30T10:41:02.000Z" "2009-10-30T00:58:18.000Z"
#> [21] "2009-10-29T19:57:28.000Z" "2009-10-29T19:49:54.000Z"
#> [23] "2009-10-29T19:07:31.000Z" "2009-10-29T19:02:50.000Z"
#> [25] "2009-10-29T18:16:44.000Z" "2009-10-29T17:29:53.000Z"
#> [27] "2009-10-29T16:59:07.000Z" "2009-10-29T16:55:30.000Z"
#> [29] "2009-10-29T16:39:48.000Z" "2009-10-29T16:03:27.000Z"
#> [31] "2009-10-29T14:38:29.000Z" "2009-10-29T12:34:56.000Z"
#> [33] "2009-10-29T12:09:01.000Z" "2009-10-29T11:07:12.000Z"
#> [35] "2009-10-29T10:32:43.000Z" "2009-10-29T04:07:01.000Z"
#> [37] "2009-10-28T22:05:47.000Z" "2009-10-28T22:02:24.000Z"
#> [39] "2009-10-28T20:16:26.000Z" "2009-10-28T17:05:45.000Z"
#> [41] "2009-10-28T16:35:34.000Z" "2009-10-28T15:56:34.000Z"
#> [43] "2009-10-28T15:53:45.000Z" "2009-10-28T14:58:05.000Z"
#> [45] "2009-10-28T13:48:42.000Z" "2009-10-28T12:52:27.000Z"
#> [47] "2009-10-28T12:47:13.000Z" "2009-10-28T09:52:45.000Z"
#> [49] "2009-10-28T04:18:01.000Z" "2009-10-27T19:42:21.000Z"
#> [51] "2009-10-27T19:39:48.000Z" "2009-10-27T19:06:10.000Z"
#> [53] "2009-10-27T18:58:14.000Z" "2009-10-27T18:35:58.000Z"
#> [55] "2009-10-27T17:43:26.000Z" "2009-10-27T17:26:42.000Z"
#> [57] "2009-10-27T17:03:08.000Z" "2009-10-27T17:01:06.000Z"
#> [59] "2009-10-27T16:35:37.000Z" "2009-10-27T16:05:40.000Z"
#> [61] "2009-10-27T15:09:22.000Z" "2009-10-27T15:08:07.000Z"
#> [63] "2009-10-27T14:42:48.000Z" "2009-10-27T14:25:39.000Z"
#> [65] "2009-10-27T13:46:23.000Z" "2009-10-27T12:54:39.000Z"
#> [67] "2009-10-27T12:28:23.000Z" "2009-10-27T10:17:42.000Z"
#> [69] "2009-10-26T21:17:07.000Z" "2009-10-26T20:19:23.000Z"
#> [71] "2009-10-26T20:17:04.000Z" "2009-10-26T20:09:13.000Z"
#> [73] "2009-10-26T19:07:11.000Z" "2009-10-26T18:32:38.000Z"
#> [75] "2009-10-26T17:50:01.000Z" "2009-10-26T17:16:45.000Z"
#> [77] "2009-10-26T15:49:27.000Z" "2009-10-26T15:05:28.000Z"
#> [79] "2009-10-26T15:00:52.000Z" "2009-10-26T13:09:21.000Z"
#> [81] "2009-10-26T12:48:58.000Z" "2009-10-26T12:31:21.000Z"
#> [83] "2009-10-26T12:27:49.000Z" "2009-10-26T10:31:56.000Z"
#> [85] "2009-10-25T12:12:41.000Z" "2009-10-24T14:54:21.000Z"
#> [87] "2009-10-24T11:30:31.000Z" "2009-10-23T21:54:18.000Z"
#> [89] "2009-10-23T20:56:29.000Z" "2009-10-23T19:09:46.000Z"
#> [91] "2009-10-23T18:44:09.000Z" "2009-10-23T18:15:07.000Z"
#> [93] "2009-10-23T18:12:14.000Z" "2009-10-23T16:06:49.000Z"
#> [95] "2009-10-23T15:43:54.000Z" "2009-10-23T15:38:07.000Z"
#> [97] "2009-10-23T15:34:26.000Z" "2009-10-23T13:52:58.000Z"
#> [99] "2009-10-23T13:51:43.000Z" "2009-10-23T12:51:54.000Z"
#> [101] "2009-10-23T12:22:59.000Z" "2009-10-23T11:09:51.000Z"
#> [103] "2009-10-22T21:39:03.000Z" "2009-10-22T20:55:41.000Z"
#> [105] "2009-10-22T20:06:15.000Z" "2009-10-22T18:37:38.000Z"
#> [107] "2009-10-22T18:28:09.000Z" "2009-10-22T17:46:23.000Z"
#> [109] "2009-10-22T17:35:03.000Z" "2009-10-22T17:31:02.000Z"
#> [111] "2009-10-22T17:30:07.000Z" "2009-10-22T16:47:16.000Z"
#> [113] "2009-10-22T15:24:51.000Z" "2009-10-22T13:02:41.000Z"
#> [115] "2009-10-22T12:43:04.000Z" "2009-10-22T11:08:39.000Z"
#> [117] "2009-10-21T21:25:50.000Z" "2009-10-21T20:27:49.000Z"
#> [119] "2009-10-21T20:22:15.000Z" "2009-10-21T20:14:15.000Z"
#> [121] "2009-10-21T17:39:20.000Z" "2009-10-21T14:57:53.000Z"
#> [123] "2009-10-21T14:16:02.000Z" "2009-10-21T13:58:19.000Z"
#> [125] "2009-10-21T13:10:56.000Z" "2009-10-21T12:54:52.000Z"
#> [127] "2009-10-21T10:08:35.000Z" "2009-10-21T09:51:32.000Z"
#> [129] "2009-10-21T01:43:09.000Z" "2009-10-20T20:59:59.000Z"
#> [131] "2009-10-20T20:09:22.000Z" "2009-10-20T19:04:11.000Z"
#> [133] "2009-10-20T18:21:37.000Z" "2009-10-20T17:39:39.000Z"
#> [135] "2009-10-20T16:47:25.000Z" "2009-10-20T15:53:02.000Z"
#> [137] "2009-10-20T15:29:59.000Z" "2009-10-20T15:00:32.000Z"
#> [139] "2009-10-20T14:00:29.000Z" "2009-10-20T12:50:32.000Z"
#> [141] "2009-10-20T12:27:52.000Z" "2009-10-20T10:19:23.000Z"
#> [143] "2009-10-19T21:36:32.000Z" "2009-10-19T21:07:00.000Z"
#> [145] "2009-10-19T21:03:24.000Z" "2009-10-19T20:07:50.000Z"
#> [147] "2009-10-19T19:25:14.000Z" "2009-10-19T19:12:14.000Z"
#> [149] "2009-10-19T17:04:42.000Z" "2009-10-19T16:30:28.000Z"
#> [151] "2009-10-19T15:07:16.000Z" "2009-10-19T15:03:57.000Z"
#> [153] "2009-10-19T14:36:32.000Z" "2009-10-19T13:44:19.000Z"
#> [155] "2009-10-19T13:35:08.000Z" "2009-10-19T12:31:28.000Z"
#> [157] "2009-10-19T10:53:04.000Z" "2009-10-18T17:19:34.000Z"
#> [159] "2009-10-18T12:23:41.000Z" "2009-10-18T00:56:22.000Z"
#> [161] "2009-10-17T16:30:23.000Z" "2009-10-17T15:42:11.000Z"
#> [163] "2009-10-17T14:53:54.000Z" "2009-10-17T11:35:04.000Z"
#> [165] "2009-10-17T00:20:11.000Z" "2009-10-16T21:02:29.000Z"
#> [167] "2009-10-16T20:04:52.000Z" "2009-10-16T17:15:49.000Z"
#> [169] "2009-10-16T15:28:48.000Z" "2009-10-16T14:47:08.000Z"
#> [171] "2009-10-16T14:45:57.000Z" "2009-10-16T13:58:51.000Z"
#> [173] "2009-10-16T13:24:28.000Z" "2009-10-16T13:13:25.000Z"
#> [175] "2009-10-16T12:20:43.000Z" "2009-10-16T11:12:12.000Z"
#> [177] "2009-10-16T10:45:07.000Z" "2009-10-15T19:44:15.000Z"
#> [179] "2009-10-15T19:26:22.000Z" "2009-10-15T18:30:44.000Z"
#> [181] "2009-10-15T17:47:48.000Z" "2009-10-15T17:46:59.000Z"
#> [183] "2009-10-15T15:08:08.000Z" "2009-10-15T14:08:14.000Z"
#> [185] "2009-10-15T13:27:58.000Z" "2009-10-15T12:16:00.000Z"
#> [187] "2009-10-15T10:35:37.000Z" "2009-10-15T10:24:44.000Z"
#> [189] "2009-10-14T21:35:07.000Z" "2009-10-14T20:44:14.000Z"
#> [191] "2009-10-14T20:38:45.000Z" "2009-10-14T20:37:12.000Z"
#> [193] "2009-10-14T20:35:47.000Z" "2009-10-14T18:08:06.000Z"
#> [195] "2009-10-14T16:35:05.000Z" "2009-10-14T16:33:40.000Z"
#> [197] "2009-10-14T15:04:18.000Z" "2009-10-14T12:44:07.000Z"
#> [199] "2009-10-14T12:43:20.000Z" "2009-10-14T12:42:53.000Z"
#> [201] "2009-10-14T12:42:25.000Z" "2009-10-14T12:41:49.000Z"
#> [203] "2009-10-13T22:14:02.000Z" "2009-10-13T22:09:34.000Z"
#> [205] "2009-10-13T22:07:20.000Z" "2009-10-13T17:36:14.000Z"
#> [207] "2009-10-13T17:34:19.000Z" "2009-10-13T17:32:36.000Z"
#> [209] "2009-10-13T12:02:46.000Z" "2009-10-13T12:01:50.000Z"
#> [211] "2009-10-13T12:00:33.000Z" "2009-10-13T12:00:21.000Z"
#> [213] "2009-10-13T11:58:46.000Z" "2009-10-13T11:56:47.000Z"
#> [215] "2009-10-13T11:56:05.000Z" "2009-10-12T18:43:55.000Z"
#> [217] "2009-10-12T18:37:03.000Z" "2009-10-12T18:35:13.000Z"
#> [219] "2009-10-09T16:11:28.000Z" "2009-10-09T16:10:12.000Z"
#> [221] "2009-10-09T16:08:01.000Z" "2009-10-09T16:06:47.000Z"
#> [223] "2009-10-09T16:05:32.000Z" "2009-10-09T15:16:25.000Z"
#> [225] "2009-10-09T13:40:15.000Z" "2009-10-09T13:38:40.000Z"
#> [227] "2009-10-09T11:15:38.000Z" "2009-10-09T10:24:41.000Z"
#> [229] "2009-10-09T10:21:21.000Z" "2009-10-09T10:20:53.000Z"
#> [231] "2009-10-09T10:20:06.000Z" "2009-10-09T10:19:30.000Z"
#> [233] "2009-10-09T10:19:14.000Z" "2009-10-08T21:55:16.000Z"
#> [235] "2009-10-08T15:58:24.000Z" "2009-10-08T15:11:07.000Z"
#> [237] "2009-10-08T15:06:14.000Z" "2009-10-08T15:01:37.000Z"
#> [239] "2009-10-08T11:21:24.000Z" "2009-10-08T11:20:06.000Z"
#> [241] "2009-10-08T11:18:49.000Z" "2009-10-08T11:17:42.000Z"
#> [243] "2009-10-08T11:17:05.000Z" "2009-10-07T22:35:25.000Z"
#> [245] "2009-10-07T22:33:53.000Z" "2009-10-07T22:31:32.000Z"
#> [247] "2009-10-07T17:57:13.000Z" "2009-10-07T16:01:10.000Z"
#> [249] "2009-10-07T15:41:17.000Z" "2009-10-07T15:38:52.000Z"
#> [251] "2009-10-07T15:36:06.000Z" "2009-10-07T15:34:12.000Z"
#> [253] "2009-10-07T11:20:00.000Z" "2009-10-07T11:18:25.000Z"
#> [255] "2009-10-07T11:16:54.000Z" "2009-10-07T11:16:28.000Z"
#> [257] "2009-10-07T11:15:39.000Z" "2009-10-07T09:56:55.000Z"
#> [259] "2009-10-07T03:50:01.000Z" "2009-10-07T03:43:53.000Z"
#> [261] "2009-10-07T02:39:55.000Z" "2009-10-07T01:33:12.000Z"
#> [263] "2009-10-07T01:21:59.000Z" "2009-10-06T20:46:33.000Z"
#> [265] "2009-10-06T14:28:48.000Z" "2009-10-06T11:20:43.000Z"
#> [267] "2009-10-06T11:20:01.000Z" "2009-10-06T11:19:06.000Z"
#> [269] "2009-10-06T11:16:27.000Z" "2009-10-06T11:15:10.000Z"
#> [271] "2009-10-06T11:13:56.000Z" "2009-10-06T09:58:24.000Z"
#> [273] "2009-10-05T16:10:51.000Z" "2009-10-05T11:33:00.000Z"
#> [275] "2009-10-05T11:32:08.000Z" "2009-10-05T11:30:31.000Z"
#> [277] "2009-10-05T11:29:06.000Z" "2009-10-05T09:51:10.000Z"
#> [279] "2009-10-04T18:18:47.000Z" "2009-10-04T18:16:25.000Z"
#> [281] "2009-10-04T18:14:51.000Z" "2009-10-04T18:13:41.000Z"
#> [283] "2009-10-04T18:11:42.000Z" "2009-10-04T03:28:31.000Z"
#> [285] "2009-10-04T03:04:46.000Z" "2009-10-04T01:32:30.000Z"
#> [287] "2009-10-04T00:41:34.000Z" "2009-10-03T16:11:31.000Z"
#> [289] "2009-10-03T16:09:05.000Z" "2009-10-03T16:07:38.000Z"
#> [291] "2009-10-03T16:06:15.000Z" "2009-10-03T16:04:49.000Z"
#> [293] "2009-10-02T20:48:07.000Z" "2009-10-02T20:45:45.000Z"
#> [295] "2009-10-02T16:51:06.000Z" "2009-10-02T15:29:21.000Z"
#> [297] "2009-10-02T12:33:38.000Z" "2009-10-02T11:06:38.000Z"
#> [299] "2009-10-02T11:05:16.000Z" "2009-10-02T11:04:41.000Z"
#> [301] "2009-10-02T11:04:01.000Z" "2009-10-02T11:03:35.000Z"
#> [303] "2009-10-02T08:22:38.000Z" "2009-10-01T22:39:52.000Z"
#> [305] "2009-10-01T21:50:11.000Z" "2009-10-01T18:30:11.000Z"
#> [307] "2009-10-01T18:27:32.000Z" "2009-10-01T18:20:20.000Z"
#> [309] "2009-10-01T18:18:20.000Z" "2009-10-01T14:38:13.000Z"
#> [311] "2009-10-01T11:11:33.000Z" "2009-10-01T11:08:56.000Z"
#> [313] "2009-10-01T11:07:56.000Z" "2009-10-01T11:07:28.000Z"
#> [315] "2009-10-01T11:06:40.000Z" "2009-10-01T02:43:50.000Z"
#> [317] "2009-09-30T21:14:21.000Z" "2009-09-30T18:07:09.000Z"
#> [319] "2009-09-30T17:11:30.000Z" "2009-09-30T14:08:16.000Z"
#> [321] "2009-09-30T11:13:50.000Z" "2009-09-30T11:13:11.000Z"
#> [323] "2009-09-30T11:12:25.000Z" "2009-09-30T11:11:59.000Z"
#> [325] "2009-09-30T11:10:10.000Z" "2009-09-30T11:09:17.000Z"
#> [327] "2009-09-30T11:08:24.000Z" "2009-09-29T21:02:22.000Z"
#> [329] "2009-09-29T19:54:53.000Z" "2009-09-29T19:07:25.000Z"
#> [331] "2009-09-29T17:43:26.000Z" "2009-09-29T11:12:07.000Z"
#> [333] "2009-09-29T11:11:41.000Z" "2009-09-29T11:11:17.000Z"
#> [335] "2009-09-29T11:10:49.000Z" "2009-09-29T11:10:03.000Z"
#> [337] "2009-09-29T11:08:40.000Z" "2009-09-28T21:22:42.000Z"
#> [339] "2009-09-28T21:16:21.000Z" "2009-09-28T20:09:06.000Z"
#> [341] "2009-09-28T20:06:12.000Z" "2009-09-28T20:03:18.000Z"
#> [343] "2009-09-28T20:02:13.000Z" "2009-09-28T19:16:44.000Z"
#> [345] "2009-09-28T11:28:35.000Z" "2009-09-28T11:22:16.000Z"
#> [347] "2009-09-28T11:19:26.000Z" "2009-09-28T11:17:22.000Z"
#> [349] "2009-09-28T11:15:09.000Z" "2009-09-28T11:10:41.000Z"
#> [351] "2009-09-28T11:08:14.000Z" "2009-09-28T11:05:15.000Z"
#> [353] "2009-09-28T11:00:20.000Z" "2009-09-27T18:56:51.000Z"
#> [355] "2009-09-27T18:52:57.000Z" "2009-09-27T15:50:20.000Z"
#> [357] "2009-09-27T15:48:14.000Z" "2009-09-27T15:43:52.000Z"
#> [359] "2009-09-27T15:41:03.000Z" "2009-09-25T19:17:37.000Z"
#> [361] "2009-09-25T19:12:56.000Z" "2009-09-25T19:09:44.000Z"
#> [363] "2009-09-25T19:07:04.000Z" "2009-09-25T14:29:10.000Z"
#> [365] "2009-09-25T10:45:38.000Z" "2009-09-25T07:47:49.000Z"
#> [367] "2009-09-25T07:45:51.000Z" "2009-09-25T07:43:09.000Z"
#> [369] "2009-09-25T07:41:25.000Z" "2009-09-25T07:39:29.000Z"
#> [371] "2009-09-25T05:36:52.000Z" "2009-09-25T01:30:36.000Z"
#> [373] "2009-09-25T00:21:58.000Z" "2009-09-24T23:58:18.000Z"
#> [375] "2009-09-24T23:05:24.000Z" "2009-09-24T19:47:21.000Z"
#> [377] "2009-09-24T19:45:30.000Z" "2009-09-24T19:32:15.000Z"
#> [379] "2009-09-24T17:49:49.000Z" "2009-09-24T16:01:54.000Z"
#> [381] "2009-09-24T15:39:30.000Z" "2009-09-24T15:36:56.000Z"
#> [383] "2009-09-24T11:34:36.000Z" "2009-09-24T11:29:45.000Z"
#> [385] "2009-09-24T11:23:07.000Z" "2009-09-24T11:20:16.000Z"
#> [387] "2009-09-24T11:18:24.000Z" "2009-09-24T11:16:44.000Z"
#> [389] "2009-09-23T22:22:28.000Z" "2009-09-23T21:36:50.000Z"
#> [391] "2009-09-23T21:34:35.000Z" "2009-09-23T21:19:46.000Z"
#> [393] "2009-09-23T21:13:58.000Z" "2009-09-23T20:36:43.000Z"
#> [395] "2009-09-23T18:21:45.000Z" "2009-09-23T17:58:13.000Z"
#> [397] "2009-09-23T17:55:49.000Z" "2009-09-23T17:52:43.000Z"
#> [399] "2009-09-23T15:43:35.000Z" "2009-09-23T11:31:16.000Z"
#> [401] "2009-09-23T11:29:29.000Z" "2009-09-23T11:27:04.000Z"
#> [403] "2009-09-23T11:24:42.000Z" "2009-09-23T11:20:25.000Z"
#> [405] "2009-09-23T04:02:52.000Z" "2009-09-23T00:42:16.000Z"
#> [407] "2009-09-22T23:50:44.000Z" "2009-09-22T20:28:33.000Z"
#> [409] "2009-09-22T18:44:53.000Z" "2009-09-22T17:32:20.000Z"
#> [411] "2009-09-22T17:29:52.000Z" "2009-09-22T16:40:30.000Z"
#> [413] "2009-09-22T15:14:57.000Z" "2009-09-22T15:08:09.000Z"
#> [415] "2009-09-22T14:07:26.000Z" "2009-09-22T13:12:38.000Z"
#> [417] "2009-09-22T11:34:25.000Z" "2009-09-22T11:33:22.000Z"
#> [419] "2009-09-22T09:53:49.000Z" "2009-09-21T21:29:20.000Z"
#> [421] "2009-09-21T19:29:55.000Z" "2009-09-21T19:16:51.000Z"
#> [423] "2009-09-21T18:26:24.000Z" "2009-09-21T18:25:20.000Z"
#> [425] "2009-09-21T18:13:24.000Z" "2009-09-21T18:05:43.000Z"
#> [427] "2009-09-21T17:41:09.000Z" "2009-09-21T16:39:02.000Z"
#> [429] "2009-09-21T16:23:42.000Z" "2009-09-21T14:53:25.000Z"
#> [431] "2009-09-21T14:50:44.000Z" "2009-09-21T14:30:21.000Z"
#> [433] "2009-09-21T13:06:07.000Z" "2009-09-21T09:59:33.000Z"
#> [435] "2009-09-21T09:58:24.000Z" "2009-09-21T09:53:48.000Z"
#> [437] "2009-09-21T04:07:37.000Z" "2009-09-21T03:02:08.000Z"
#> [439] "2009-09-21T02:56:00.000Z" "2009-09-21T02:51:36.000Z"
#> [441] "2009-09-21T02:42:16.000Z" "2009-09-21T02:10:13.000Z"
#> [443] "2009-09-21T00:52:34.000Z" "2009-09-21T00:52:18.000Z"
#> [445] "2009-09-21T00:38:31.000Z" "2009-09-20T14:43:54.000Z"
#> [447] "2009-09-20T11:30:56.000Z" "2009-09-20T11:28:28.000Z"
#> [449] "2009-09-19T11:20:08.000Z" "2009-09-19T11:18:09.000Z"
#> [451] "2009-09-19T04:06:19.000Z" "2009-09-18T20:45:38.000Z"
#> [453] "2009-09-18T19:31:41.000Z" "2009-09-18T18:59:43.000Z"
#> [455] "2009-09-18T18:43:09.000Z" "2009-09-18T17:28:28.000Z"
#> [457] "2009-09-18T15:57:09.000Z" "2009-09-18T14:15:31.000Z"
#> [459] "2009-09-18T12:54:43.000Z" "2009-09-18T10:25:04.000Z"
#> [461] "2009-09-18T09:55:57.000Z" "2009-09-17T19:55:50.000Z"
#> [463] "2009-09-17T19:41:13.000Z" "2009-09-17T19:37:30.000Z"
#> [465] "2009-09-17T19:35:41.000Z" "2009-09-17T17:39:58.000Z"
#> [467] "2009-09-17T16:40:41.000Z" "2009-09-17T16:18:20.000Z"
#> [469] "2009-09-17T15:20:55.000Z" "2009-09-17T13:30:14.000Z"
#> [471] "2009-09-17T12:40:05.000Z" "2009-09-17T11:12:23.000Z"
#> [473] "2009-09-17T10:39:40.000Z" "2009-09-17T05:51:45.000Z"
#> [475] "2009-09-16T19:24:18.000Z" "2009-09-16T19:06:11.000Z"
#> [477] "2009-09-16T17:28:45.000Z" "2009-09-16T16:23:03.000Z"
#> [479] "2009-09-16T14:11:32.000Z" "2009-09-16T13:51:08.000Z"
#> [481] "2009-09-16T13:43:38.000Z" "2009-09-16T13:25:34.000Z"
#> [483] "2009-09-16T13:25:04.000Z" "2009-09-16T13:23:29.000Z"
#> [485] "2009-09-16T13:17:39.000Z" "2009-09-16T13:04:45.000Z"
#> [487] "2009-09-16T09:59:02.000Z" "2009-09-16T09:54:52.000Z"
#> [489] "2009-09-15T21:34:43.000Z" "2009-09-15T21:10:52.000Z"
#> [491] "2009-09-15T20:34:38.000Z" "2009-09-15T19:39:47.000Z"
#> [493] "2009-09-15T18:21:53.000Z" "2009-09-15T18:20:33.000Z"
#> [495] "2009-09-15T18:17:22.000Z" "2009-09-15T17:26:32.000Z"
#> [497] "2009-09-15T17:25:14.000Z" "2009-09-15T15:36:55.000Z"
#> [499] "2009-09-15T15:09:24.000Z" "2009-09-15T13:45:39.000Z"
nrow(wapo_ft)
#> [1] 500
wapo_ft2 <- get_all_tweets(users = "washingtonpost", start_tweets = "2009-08-01T00:00:00Z", end_tweets = "2009-09-01T00:00:00Z", is_retweet=FALSE, is_reply=FALSE, n = 1000, verbose = FALSE)
nrow(wapo_ft2)
#> [1] 313
wapo_ft3 <- get_all_tweets(users = "washingtonpost", start_tweets = "2009-08-01T00:00:00Z", end_tweets = "2009-08-04T00:00:00Z", is_retweet=FALSE, is_reply=FALSE, n = 1000, verbose = FALSE)
nrow(wapo_ft3)
#> [1] 17 Created on 2022-04-17 by the reprex package (v2.0.1) |
Beta Was this translation helpful? Give feedback.
-
awesome. Thank you for the sanity check. i'll download them separately and
combine.
Respectfully,
John Bolton
***@***.***
402-598-5165
Zoom: https://JHUBlueJays.zoom.us/j/6597993706
…On Sun, Apr 17, 2022 at 11:00 AM Chung-hong Chan ***@***.***> wrote:
@jqb101 <https://github.com/jqb101> Major, I know that you have asked the
same question on Twitter Developer Community and the developer of twarc2
has given you some advice. The developer of twarc tried it only with the
count endpoint.
Here I tried both the count and the search end points. What I want to tell
you is that it is not likely to be an issue with the package. It can indeed
get the data from the time range you mentioned. Instead, the Twitter search
index is very wonky with old tweets. My suggestion is not to collect your
tweets in one query but to break it down into multiple queries and explore
what went wrong.
require(academictwitteR)#> Loading required package: academictwitteR
wapo_counts <- count_all_tweets(users = "washingtonpost", start_tweets = "2009-08-01T00:00:00Z", end_tweets = "2009-10-31T00:00:00Z", is_retweet=FALSE, is_reply=FALSE, n = 300)#> query: (from:washingtonpost) -is:retweet -is:reply #> Total pages queried: 1 (tweets captured this page: 31).#> Total pages queried: 2 (tweets captured this page: 31).#> Total pages queried: 3 (tweets captured this page: 29).#> This is the last page for (from:washingtonpost) -is:retweet -is:reply : finishing collection.
sum(wapo_counts$tweet_count)#> [1] 956
wapo_ft <- get_all_tweets(users = "washingtonpost", start_tweets = "2009-08-01T00:00:00Z", end_tweets = "2009-10-31T00:00:00Z", is_retweet=FALSE, is_reply=FALSE, n = 1000, verbose = FALSE)
wapo_ft$created_at#> [1] "2009-10-30T23:05:46.000Z" "2009-10-30T21:16:30.000Z"#> [3] "2009-10-30T20:43:13.000Z" "2009-10-30T20:04:56.000Z"#> [5] "2009-10-30T19:35:23.000Z" "2009-10-30T18:28:17.000Z"#> [7] "2009-10-30T18:08:17.000Z" "2009-10-30T18:03:19.000Z"#> [9] "2009-10-30T17:08:14.000Z" "2009-10-30T15:53:29.000Z"#> [11] "2009-10-30T15:47:15.000Z" "2009-10-30T14:51:28.000Z"#> [13] "2009-10-30T13:59:50.000Z" "2009-10-30T13:51:52.000Z"#> [15] "2009-10-30T13:25:33.000Z" "2009-10-30T13:04:56.000Z"#> [17] "2009-10-30T12:44:31.000Z" "2009-10-30T12:43:52.000Z"#> [19] "2009-10-30T10:41:02.000Z" "2009-10-30T00:58:18.000Z"#> [21] "2009-10-29T19:57:28.000Z" "2009-10-29T19:49:54.000Z"#> [23] "2009-10-29T19:07:31.000Z" "2009-10-29T19:02:50.000Z"#> [25] "2009-10-29T18:16:44.000Z" "2009-10-29T17:29:53.000Z"#> [27] "2009-10-29T16:59:07.000Z" "2009-10-29T16:55:30.000Z"#> [29] "2009-10-29T16:39:48.000Z" "2009-10-29T16:03:27.000Z"#> [31] "2009-10-29T14:38:29.000Z" "2009-10-29T12:34:56.000Z"#> [33] "2009-10-29T12:09:01.000Z" "2009-10-29T11:07:12.000Z"#> [35] "2009-10-29T10:32:43.000Z" "2009-10-29T04:07:01.000Z"#> [37] "2009-10-28T22:05:47.000Z" "2009-10-28T22:02:24.000Z"#> [39] "2009-10-28T20:16:26.000Z" "2009-10-28T17:05:45.000Z"#> [41] "2009-10-28T16:35:34.000Z" "2009-10-28T15:56:34.000Z"#> [43] "2009-10-28T15:53:45.000Z" "2009-10-28T14:58:05.000Z"#> [45] "2009-10-28T13:48:42.000Z" "2009-10-28T12:52:27.000Z"#> [47] "2009-10-28T12:47:13.000Z" "2009-10-28T09:52:45.000Z"#> [49] "2009-10-28T04:18:01.000Z" "2009-10-27T19:42:21.000Z"#> [51] "2009-10-27T19:39:48.000Z" "2009-10-27T19:06:10.000Z"#> [53] "2009-10-27T18:58:14.000Z" "2009-10-27T18:35:58.000Z"#> [55] "2009-10-27T17:43:26.000Z" "2009-10-27T17:26:42.000Z"#> [57] "2009-10-27T17:03:08.000Z" "2009-10-27T17:01:06.000Z"#> [59] "2009-10-27T16:35:37.000Z" "2009-10-27T16:05:40.000Z"#> [61] "2009-10-27T15:09:22.000Z" "2009-10-27T15:08:07.000Z"#> [63] "2009-10-27T14:42:48.000Z" "2009-10-27T14:25:39.000Z"#> [65] "2009-10-27T13:46:23.000Z" "2009-10-27T12:54:39.000Z"#> [67] "2009-10-27T12:28:23.000Z" "2009-10-27T10:17:42.000Z"#> [69] "2009-10-26T21:17:07.000Z" "2009-10-26T20:19:23.000Z"#> [71] "2009-10-26T20:17:04.000Z" "2009-10-26T20:09:13.000Z"#> [73] "2009-10-26T19:07:11.000Z" "2009-10-26T18:32:38.000Z"#> [75] "2009-10-26T17:50:01.000Z" "2009-10-26T17:16:45.000Z"#> [77] "2009-10-26T15:49:27.000Z" "2009-10-26T15:05:28.000Z"#> [79] "2009-10-26T15:00:52.000Z" "2009-10-26T13:09:21.000Z"#> [81] "2009-10-26T12:48:58.000Z" "2009-10-26T12:31:21.000Z"#> [83] "2009-10-26T12:27:49.000Z" "2009-10-26T10:31:56.000Z"#> [85] "2009-10-25T12:12:41.000Z" "2009-10-24T14:54:21.000Z"#> [87] "2009-10-24T11:30:31.000Z" "2009-10-23T21:54:18.000Z"#> [89] "2009-10-23T20:56:29.000Z" "2009-10-23T19:09:46.000Z"#> [91] "2009-10-23T18:44:09.000Z" "2009-10-23T18:15:07.000Z"#> [93] "2009-10-23T18:12:14.000Z" "2009-10-23T16:06:49.000Z"#> [95] "2009-10-23T15:43:54.000Z" "2009-10-23T15:38:07.000Z"#> [97] "2009-10-23T15:34:26.000Z" "2009-10-23T13:52:58.000Z"#> [99] "2009-10-23T13:51:43.000Z" "2009-10-23T12:51:54.000Z"#> [101] "2009-10-23T12:22:59.000Z" "2009-10-23T11:09:51.000Z"#> [103] "2009-10-22T21:39:03.000Z" "2009-10-22T20:55:41.000Z"#> [105] "2009-10-22T20:06:15.000Z" "2009-10-22T18:37:38.000Z"#> [107] "2009-10-22T18:28:09.000Z" "2009-10-22T17:46:23.000Z"#> [109] "2009-10-22T17:35:03.000Z" "2009-10-22T17:31:02.000Z"#> [111] "2009-10-22T17:30:07.000Z" "2009-10-22T16:47:16.000Z"#> [113] "2009-10-22T15:24:51.000Z" "2009-10-22T13:02:41.000Z"#> [115] "2009-10-22T12:43:04.000Z" "2009-10-22T11:08:39.000Z"#> [117] "2009-10-21T21:25:50.000Z" "2009-10-21T20:27:49.000Z"#> [119] "2009-10-21T20:22:15.000Z" "2009-10-21T20:14:15.000Z"#> [121] "2009-10-21T17:39:20.000Z" "2009-10-21T14:57:53.000Z"#> [123] "2009-10-21T14:16:02.000Z" "2009-10-21T13:58:19.000Z"#> [125] "2009-10-21T13:10:56.000Z" "2009-10-21T12:54:52.000Z"#> [127] "2009-10-21T10:08:35.000Z" "2009-10-21T09:51:32.000Z"#> [129] "2009-10-21T01:43:09.000Z" "2009-10-20T20:59:59.000Z"#> [131] "2009-10-20T20:09:22.000Z" "2009-10-20T19:04:11.000Z"#> [133] "2009-10-20T18:21:37.000Z" "2009-10-20T17:39:39.000Z"#> [135] "2009-10-20T16:47:25.000Z" "2009-10-20T15:53:02.000Z"#> [137] "2009-10-20T15:29:59.000Z" "2009-10-20T15:00:32.000Z"#> [139] "2009-10-20T14:00:29.000Z" "2009-10-20T12:50:32.000Z"#> [141] "2009-10-20T12:27:52.000Z" "2009-10-20T10:19:23.000Z"#> [143] "2009-10-19T21:36:32.000Z" "2009-10-19T21:07:00.000Z"#> [145] "2009-10-19T21:03:24.000Z" "2009-10-19T20:07:50.000Z"#> [147] "2009-10-19T19:25:14.000Z" "2009-10-19T19:12:14.000Z"#> [149] "2009-10-19T17:04:42.000Z" "2009-10-19T16:30:28.000Z"#> [151] "2009-10-19T15:07:16.000Z" "2009-10-19T15:03:57.000Z"#> [153] "2009-10-19T14:36:32.000Z" "2009-10-19T13:44:19.000Z"#> [155] "2009-10-19T13:35:08.000Z" "2009-10-19T12:31:28.000Z"#> [157] "2009-10-19T10:53:04.000Z" "2009-10-18T17:19:34.000Z"#> [159] "2009-10-18T12:23:41.000Z" "2009-10-18T00:56:22.000Z"#> [161] "2009-10-17T16:30:23.000Z" "2009-10-17T15:42:11.000Z"#> [163] "2009-10-17T14:53:54.000Z" "2009-10-17T11:35:04.000Z"#> [165] "2009-10-17T00:20:11.000Z" "2009-10-16T21:02:29.000Z"#> [167] "2009-10-16T20:04:52.000Z" "2009-10-16T17:15:49.000Z"#> [169] "2009-10-16T15:28:48.000Z" "2009-10-16T14:47:08.000Z"#> [171] "2009-10-16T14:45:57.000Z" "2009-10-16T13:58:51.000Z"#> [173] "2009-10-16T13:24:28.000Z" "2009-10-16T13:13:25.000Z"#> [175] "2009-10-16T12:20:43.000Z" "2009-10-16T11:12:12.000Z"#> [177] "2009-10-16T10:45:07.000Z" "2009-10-15T19:44:15.000Z"#> [179] "2009-10-15T19:26:22.000Z" "2009-10-15T18:30:44.000Z"#> [181] "2009-10-15T17:47:48.000Z" "2009-10-15T17:46:59.000Z"#> [183] "2009-10-15T15:08:08.000Z" "2009-10-15T14:08:14.000Z"#> [185] "2009-10-15T13:27:58.000Z" "2009-10-15T12:16:00.000Z"#> [187] "2009-10-15T10:35:37.000Z" "2009-10-15T10:24:44.000Z"#> [189] "2009-10-14T21:35:07.000Z" "2009-10-14T20:44:14.000Z"#> [191] "2009-10-14T20:38:45.000Z" "2009-10-14T20:37:12.000Z"#> [193] "2009-10-14T20:35:47.000Z" "2009-10-14T18:08:06.000Z"#> [195] "2009-10-14T16:35:05.000Z" "2009-10-14T16:33:40.000Z"#> [197] "2009-10-14T15:04:18.000Z" "2009-10-14T12:44:07.000Z"#> [199] "2009-10-14T12:43:20.000Z" "2009-10-14T12:42:53.000Z"#> [201] "2009-10-14T12:42:25.000Z" "2009-10-14T12:41:49.000Z"#> [203] "2009-10-13T22:14:02.000Z" "2009-10-13T22:09:34.000Z"#> [205] "2009-10-13T22:07:20.000Z" "2009-10-13T17:36:14.000Z"#> [207] "2009-10-13T17:34:19.000Z" "2009-10-13T17:32:36.000Z"#> [209] "2009-10-13T12:02:46.000Z" "2009-10-13T12:01:50.000Z"#> [211] "2009-10-13T12:00:33.000Z" "2009-10-13T12:00:21.000Z"#> [213] "2009-10-13T11:58:46.000Z" "2009-10-13T11:56:47.000Z"#> [215] "2009-10-13T11:56:05.000Z" "2009-10-12T18:43:55.000Z"#> [217] "2009-10-12T18:37:03.000Z" "2009-10-12T18:35:13.000Z"#> [219] "2009-10-09T16:11:28.000Z" "2009-10-09T16:10:12.000Z"#> [221] "2009-10-09T16:08:01.000Z" "2009-10-09T16:06:47.000Z"#> [223] "2009-10-09T16:05:32.000Z" "2009-10-09T15:16:25.000Z"#> [225] "2009-10-09T13:40:15.000Z" "2009-10-09T13:38:40.000Z"#> [227] "2009-10-09T11:15:38.000Z" "2009-10-09T10:24:41.000Z"#> [229] "2009-10-09T10:21:21.000Z" "2009-10-09T10:20:53.000Z"#> [231] "2009-10-09T10:20:06.000Z" "2009-10-09T10:19:30.000Z"#> [233] "2009-10-09T10:19:14.000Z" "2009-10-08T21:55:16.000Z"#> [235] "2009-10-08T15:58:24.000Z" "2009-10-08T15:11:07.000Z"#> [237] "2009-10-08T15:06:14.000Z" "2009-10-08T15:01:37.000Z"#> [239] "2009-10-08T11:21:24.000Z" "2009-10-08T11:20:06.000Z"#> [241] "2009-10-08T11:18:49.000Z" "2009-10-08T11:17:42.000Z"#> [243] "2009-10-08T11:17:05.000Z" "2009-10-07T22:35:25.000Z"#> [245] "2009-10-07T22:33:53.000Z" "2009-10-07T22:31:32.000Z"#> [247] "2009-10-07T17:57:13.000Z" "2009-10-07T16:01:10.000Z"#> [249] "2009-10-07T15:41:17.000Z" "2009-10-07T15:38:52.000Z"#> [251] "2009-10-07T15:36:06.000Z" "2009-10-07T15:34:12.000Z"#> [253] "2009-10-07T11:20:00.000Z" "2009-10-07T11:18:25.000Z"#> [255] "2009-10-07T11:16:54.000Z" "2009-10-07T11:16:28.000Z"#> [257] "2009-10-07T11:15:39.000Z" "2009-10-07T09:56:55.000Z"#> [259] "2009-10-07T03:50:01.000Z" "2009-10-07T03:43:53.000Z"#> [261] "2009-10-07T02:39:55.000Z" "2009-10-07T01:33:12.000Z"#> [263] "2009-10-07T01:21:59.000Z" "2009-10-06T20:46:33.000Z"#> [265] "2009-10-06T14:28:48.000Z" "2009-10-06T11:20:43.000Z"#> [267] "2009-10-06T11:20:01.000Z" "2009-10-06T11:19:06.000Z"#> [269] "2009-10-06T11:16:27.000Z" "2009-10-06T11:15:10.000Z"#> [271] "2009-10-06T11:13:56.000Z" "2009-10-06T09:58:24.000Z"#> [273] "2009-10-05T16:10:51.000Z" "2009-10-05T11:33:00.000Z"#> [275] "2009-10-05T11:32:08.000Z" "2009-10-05T11:30:31.000Z"#> [277] "2009-10-05T11:29:06.000Z" "2009-10-05T09:51:10.000Z"#> [279] "2009-10-04T18:18:47.000Z" "2009-10-04T18:16:25.000Z"#> [281] "2009-10-04T18:14:51.000Z" "2009-10-04T18:13:41.000Z"#> [283] "2009-10-04T18:11:42.000Z" "2009-10-04T03:28:31.000Z"#> [285] "2009-10-04T03:04:46.000Z" "2009-10-04T01:32:30.000Z"#> [287] "2009-10-04T00:41:34.000Z" "2009-10-03T16:11:31.000Z"#> [289] "2009-10-03T16:09:05.000Z" "2009-10-03T16:07:38.000Z"#> [291] "2009-10-03T16:06:15.000Z" "2009-10-03T16:04:49.000Z"#> [293] "2009-10-02T20:48:07.000Z" "2009-10-02T20:45:45.000Z"#> [295] "2009-10-02T16:51:06.000Z" "2009-10-02T15:29:21.000Z"#> [297] "2009-10-02T12:33:38.000Z" "2009-10-02T11:06:38.000Z"#> [299] "2009-10-02T11:05:16.000Z" "2009-10-02T11:04:41.000Z"#> [301] "2009-10-02T11:04:01.000Z" "2009-10-02T11:03:35.000Z"#> [303] "2009-10-02T08:22:38.000Z" "2009-10-01T22:39:52.000Z"#> [305] "2009-10-01T21:50:11.000Z" "2009-10-01T18:30:11.000Z"#> [307] "2009-10-01T18:27:32.000Z" "2009-10-01T18:20:20.000Z"#> [309] "2009-10-01T18:18:20.000Z" "2009-10-01T14:38:13.000Z"#> [311] "2009-10-01T11:11:33.000Z" "2009-10-01T11:08:56.000Z"#> [313] "2009-10-01T11:07:56.000Z" "2009-10-01T11:07:28.000Z"#> [315] "2009-10-01T11:06:40.000Z" "2009-10-01T02:43:50.000Z"#> [317] "2009-09-30T21:14:21.000Z" "2009-09-30T18:07:09.000Z"#> [319] "2009-09-30T17:11:30.000Z" "2009-09-30T14:08:16.000Z"#> [321] "2009-09-30T11:13:50.000Z" "2009-09-30T11:13:11.000Z"#> [323] "2009-09-30T11:12:25.000Z" "2009-09-30T11:11:59.000Z"#> [325] "2009-09-30T11:10:10.000Z" "2009-09-30T11:09:17.000Z"#> [327] "2009-09-30T11:08:24.000Z" "2009-09-29T21:02:22.000Z"#> [329] "2009-09-29T19:54:53.000Z" "2009-09-29T19:07:25.000Z"#> [331] "2009-09-29T17:43:26.000Z" "2009-09-29T11:12:07.000Z"#> [333] "2009-09-29T11:11:41.000Z" "2009-09-29T11:11:17.000Z"#> [335] "2009-09-29T11:10:49.000Z" "2009-09-29T11:10:03.000Z"#> [337] "2009-09-29T11:08:40.000Z" "2009-09-28T21:22:42.000Z"#> [339] "2009-09-28T21:16:21.000Z" "2009-09-28T20:09:06.000Z"#> [341] "2009-09-28T20:06:12.000Z" "2009-09-28T20:03:18.000Z"#> [343] "2009-09-28T20:02:13.000Z" "2009-09-28T19:16:44.000Z"#> [345] "2009-09-28T11:28:35.000Z" "2009-09-28T11:22:16.000Z"#> [347] "2009-09-28T11:19:26.000Z" "2009-09-28T11:17:22.000Z"#> [349] "2009-09-28T11:15:09.000Z" "2009-09-28T11:10:41.000Z"#> [351] "2009-09-28T11:08:14.000Z" "2009-09-28T11:05:15.000Z"#> [353] "2009-09-28T11:00:20.000Z" "2009-09-27T18:56:51.000Z"#> [355] "2009-09-27T18:52:57.000Z" "2009-09-27T15:50:20.000Z"#> [357] "2009-09-27T15:48:14.000Z" "2009-09-27T15:43:52.000Z"#> [359] "2009-09-27T15:41:03.000Z" "2009-09-25T19:17:37.000Z"#> [361] "2009-09-25T19:12:56.000Z" "2009-09-25T19:09:44.000Z"#> [363] "2009-09-25T19:07:04.000Z" "2009-09-25T14:29:10.000Z"#> [365] "2009-09-25T10:45:38.000Z" "2009-09-25T07:47:49.000Z"#> [367] "2009-09-25T07:45:51.000Z" "2009-09-25T07:43:09.000Z"#> [369] "2009-09-25T07:41:25.000Z" "2009-09-25T07:39:29.000Z"#> [371] "2009-09-25T05:36:52.000Z" "2009-09-25T01:30:36.000Z"#> [373] "2009-09-25T00:21:58.000Z" "2009-09-24T23:58:18.000Z"#> [375] "2009-09-24T23:05:24.000Z" "2009-09-24T19:47:21.000Z"#> [377] "2009-09-24T19:45:30.000Z" "2009-09-24T19:32:15.000Z"#> [379] "2009-09-24T17:49:49.000Z" "2009-09-24T16:01:54.000Z"#> [381] "2009-09-24T15:39:30.000Z" "2009-09-24T15:36:56.000Z"#> [383] "2009-09-24T11:34:36.000Z" "2009-09-24T11:29:45.000Z"#> [385] "2009-09-24T11:23:07.000Z" "2009-09-24T11:20:16.000Z"#> [387] "2009-09-24T11:18:24.000Z" "2009-09-24T11:16:44.000Z"#> [389] "2009-09-23T22:22:28.000Z" "2009-09-23T21:36:50.000Z"#> [391] "2009-09-23T21:34:35.000Z" "2009-09-23T21:19:46.000Z"#> [393] "2009-09-23T21:13:58.000Z" "2009-09-23T20:36:43.000Z"#> [395] "2009-09-23T18:21:45.000Z" "2009-09-23T17:58:13.000Z"#> [397] "2009-09-23T17:55:49.000Z" "2009-09-23T17:52:43.000Z"#> [399] "2009-09-23T15:43:35.000Z" "2009-09-23T11:31:16.000Z"#> [401] "2009-09-23T11:29:29.000Z" "2009-09-23T11:27:04.000Z"#> [403] "2009-09-23T11:24:42.000Z" "2009-09-23T11:20:25.000Z"#> [405] "2009-09-23T04:02:52.000Z" "2009-09-23T00:42:16.000Z"#> [407] "2009-09-22T23:50:44.000Z" "2009-09-22T20:28:33.000Z"#> [409] "2009-09-22T18:44:53.000Z" "2009-09-22T17:32:20.000Z"#> [411] "2009-09-22T17:29:52.000Z" "2009-09-22T16:40:30.000Z"#> [413] "2009-09-22T15:14:57.000Z" "2009-09-22T15:08:09.000Z"#> [415] "2009-09-22T14:07:26.000Z" "2009-09-22T13:12:38.000Z"#> [417] "2009-09-22T11:34:25.000Z" "2009-09-22T11:33:22.000Z"#> [419] "2009-09-22T09:53:49.000Z" "2009-09-21T21:29:20.000Z"#> [421] "2009-09-21T19:29:55.000Z" "2009-09-21T19:16:51.000Z"#> [423] "2009-09-21T18:26:24.000Z" "2009-09-21T18:25:20.000Z"#> [425] "2009-09-21T18:13:24.000Z" "2009-09-21T18:05:43.000Z"#> [427] "2009-09-21T17:41:09.000Z" "2009-09-21T16:39:02.000Z"#> [429] "2009-09-21T16:23:42.000Z" "2009-09-21T14:53:25.000Z"#> [431] "2009-09-21T14:50:44.000Z" "2009-09-21T14:30:21.000Z"#> [433] "2009-09-21T13:06:07.000Z" "2009-09-21T09:59:33.000Z"#> [435] "2009-09-21T09:58:24.000Z" "2009-09-21T09:53:48.000Z"#> [437] "2009-09-21T04:07:37.000Z" "2009-09-21T03:02:08.000Z"#> [439] "2009-09-21T02:56:00.000Z" "2009-09-21T02:51:36.000Z"#> [441] "2009-09-21T02:42:16.000Z" "2009-09-21T02:10:13.000Z"#> [443] "2009-09-21T00:52:34.000Z" "2009-09-21T00:52:18.000Z"#> [445] "2009-09-21T00:38:31.000Z" "2009-09-20T14:43:54.000Z"#> [447] "2009-09-20T11:30:56.000Z" "2009-09-20T11:28:28.000Z"#> [449] "2009-09-19T11:20:08.000Z" "2009-09-19T11:18:09.000Z"#> [451] "2009-09-19T04:06:19.000Z" "2009-09-18T20:45:38.000Z"#> [453] "2009-09-18T19:31:41.000Z" "2009-09-18T18:59:43.000Z"#> [455] "2009-09-18T18:43:09.000Z" "2009-09-18T17:28:28.000Z"#> [457] "2009-09-18T15:57:09.000Z" "2009-09-18T14:15:31.000Z"#> [459] "2009-09-18T12:54:43.000Z" "2009-09-18T10:25:04.000Z"#> [461] "2009-09-18T09:55:57.000Z" "2009-09-17T19:55:50.000Z"#> [463] "2009-09-17T19:41:13.000Z" "2009-09-17T19:37:30.000Z"#> [465] "2009-09-17T19:35:41.000Z" "2009-09-17T17:39:58.000Z"#> [467] "2009-09-17T16:40:41.000Z" "2009-09-17T16:18:20.000Z"#> [469] "2009-09-17T15:20:55.000Z" "2009-09-17T13:30:14.000Z"#> [471] "2009-09-17T12:40:05.000Z" "2009-09-17T11:12:23.000Z"#> [473] "2009-09-17T10:39:40.000Z" "2009-09-17T05:51:45.000Z"#> [475] "2009-09-16T19:24:18.000Z" "2009-09-16T19:06:11.000Z"#> [477] "2009-09-16T17:28:45.000Z" "2009-09-16T16:23:03.000Z"#> [479] "2009-09-16T14:11:32.000Z" "2009-09-16T13:51:08.000Z"#> [481] "2009-09-16T13:43:38.000Z" "2009-09-16T13:25:34.000Z"#> [483] "2009-09-16T13:25:04.000Z" "2009-09-16T13:23:29.000Z"#> [485] "2009-09-16T13:17:39.000Z" "2009-09-16T13:04:45.000Z"#> [487] "2009-09-16T09:59:02.000Z" "2009-09-16T09:54:52.000Z"#> [489] "2009-09-15T21:34:43.000Z" "2009-09-15T21:10:52.000Z"#> [491] "2009-09-15T20:34:38.000Z" "2009-09-15T19:39:47.000Z"#> [493] "2009-09-15T18:21:53.000Z" "2009-09-15T18:20:33.000Z"#> [495] "2009-09-15T18:17:22.000Z" "2009-09-15T17:26:32.000Z"#> [497] "2009-09-15T17:25:14.000Z" "2009-09-15T15:36:55.000Z"#> [499] "2009-09-15T15:09:24.000Z" "2009-09-15T13:45:39.000Z"
nrow(wapo_ft)#> [1] 500
wapo_ft2 <- get_all_tweets(users = "washingtonpost", start_tweets = "2009-08-01T00:00:00Z", end_tweets = "2009-09-01T00:00:00Z", is_retweet=FALSE, is_reply=FALSE, n = 1000, verbose = FALSE)
nrow(wapo_ft2)#> [1] 313
wapo_ft3 <- get_all_tweets(users = "washingtonpost", start_tweets = "2009-08-01T00:00:00Z", end_tweets = "2009-08-04T00:00:00Z", is_retweet=FALSE, is_reply=FALSE, n = 1000, verbose = FALSE)
nrow(wapo_ft3)#> [1] 17
Created on 2022-04-17 by the reprex package <https://reprex.tidyverse.org>
(v2.0.1)
—
Reply to this email directly, view it on GitHub
<#319 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/APTOZD362AYWCAS7BZKQPX3VFQYR5ANCNFSM5TUBVJVA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***
com>
|
Beta Was this translation helpful? Give feedback.
-
Please confirm the following
something went wrong. Status code: 400.
Describe the bug
I've been trying to pull tweets from several media companies during the Obama years for a research project. The get all tweets command has worked well but something strange as occured. The pull drops all tweets from 1AUG09-31AUG2010. I've tried multiple forms of the pull
temp<- get_all_tweets(users = media_tweets_usernames[i], start_tweets = media_tweets_start_date[i],end_tweets = media_tweets_end_date[i], bearer_token = get_bearer(),is_retweet=FALSE, is_reply=FALSE,n = 750000)
It's very strange because the pull works for everything but this period. Any advice would be helpful.
The usernames I'm using are: washingtonpost
nytimes
WSJ
FoxNews
bbcworld
FT
Huffpost
CNN
thedailybeast
msnbc
newsweek
slate
voxdotcom
AP
theatlantic
business
ABC
CBSNews
TheEconomist
guardian
NBCNews
NPR
politico
propublica
Time
YahooNews
USAToday
Reuters
axios
Diplomat_APAC
nypost
DailyCaller
FDRLST
AJEnglish
XHNews
Japantimes
BBCNewsAus
Expected Behavior
should pull tweets for the entire period but drops AUG09-AUG10
Steps To Reproduce
where media_tweets_usernames is the list of users above and start end dates are 1/20/09 through 1/20/2017 (in the proper format).
Environment
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252
system code page: 65001
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] rvest_1.0.2 scales_1.1.1 plotly_4.10.0 ggbeeswarm_0.6.0 ggalt_0.4.0 hrbrthemes_0.8.0 forcats_0.5.1
[8] RCurl_1.98-1.5 dplyr_1.0.7 Rfacebook_0.6.15 httpuv_1.6.5 tidyr_1.1.4 zoo_1.8-9 data.table_1.14.2
[15] stringi_1.7.6 servr_0.24 LDAvis_0.3.2 topicmodels_0.2-12 lubridate_1.8.0 devtools_2.4.3 usethis_2.1.5
[22] tibble_3.1.6 academictwitteR_0.3.1 twitteR_1.1.9 rtweet_0.7.0 httr_1.4.2 plyr_1.8.6 vroom_1.5.7
[29] pdftools_3.0.1 jsonlite_1.7.3 rjson_0.2.21 tidytext_0.3.2 stringr_1.4.0 ggplot2_3.3.5 syuzhet_1.0.6
[36] wordcloud_2.6 RColorBrewer_1.1-2 SnowballC_0.7.0 tm_0.7-8 NLP_0.2-1 quanteda.textstats_0.95 quanteda.textplots_0.94
[43] readtext_0.81 quanteda_3.2.0
loaded via a namespace (and not attached):
[1] colorspace_2.0-2 ellipsis_0.3.2 modeltools_0.2-23 rprojroot_2.0.2 fs_1.5.2 rstudioapi_0.13 remotes_2.4.2 bit64_4.0.5 fansi_1.0.2
[10] xml2_1.3.3 extrafont_0.17 cachem_1.0.6 knitr_1.37 pkgload_1.2.4 Rttf2pt1_1.3.9 compiler_4.1.2 lazyeval_0.2.2 assertthat_0.2.1
[19] Matrix_1.3-4 fastmap_1.1.0 cli_3.1.1 later_1.3.0 htmltools_0.5.2 prettyunits_1.1.1 tools_4.1.2 gtable_0.3.0 glue_1.6.0
[28] maps_3.4.0 fastmatch_1.1-3 Rcpp_1.0.8 slam_0.1-50 vctrs_0.3.8 extrafontdb_1.0 xfun_0.29 stopwords_2.3 ps_1.6.0
[37] brio_1.1.3 testthat_3.1.2 nsyllable_1.0 lifecycle_1.0.1 MASS_7.3-54 promises_1.2.0.1 proj4_1.0-11 parallel_4.1.2 curl_4.3.2
[46] yaml_2.2.2 memoise_2.0.1 gdtools_0.2.4 tokenizers_0.2.1 desc_1.4.0 pkgbuild_1.3.1 systemfonts_1.0.4 rlang_0.4.12 pkgconfig_2.0.3
[55] bitops_1.0-7 evaluate_0.14 lattice_0.20-45 purrr_0.3.4 htmlwidgets_1.5.4 bit_4.0.4 processx_3.5.2 tidyselect_1.1.1 magrittr_2.0.1
[64] R6_2.5.1 generics_0.1.1 DBI_1.1.2 pillar_1.6.5 withr_2.4.3 ash_1.0-15 janeaustenr_0.1.5 crayon_1.4.2 KernSmooth_2.23-20
[73] utf8_1.2.2 tzdb_0.2.0 rmarkdown_2.11 grid_4.1.2 qpdf_1.1 callr_3.7.0 digest_0.6.29 openssl_1.4.6 RcppParallel_5.1.5
[82] stats4_4.1.2 munsell_0.5.0 viridisLite_0.4.0 beeswarm_0.4.0 vipor_0.4.5 sessioninfo_1.2.2 askpass_1.1
Anything else?
No response
Beta Was this translation helpful? Give feedback.
All reactions