fennb/phirehose

[Question] Can we get previous tweet using streaming API?

Closed this issue · 2 comments

hi, sorry for my stupid question here. I read twitter streaming API, phirehose wiki and the whole source code and example. but i dont get my answer. And i ask this same question in stackoverflow but still didnt get answer until now. I need this answer before i go forward.

I work with REST API for a month. Now i decide to using STREAMING API. I try several experiment

RUN console/script
i tweet
console display my tweet

I stop console
i tweet
i immediately start console (immediately after i post tweet)
console didnt display anything

The problem is, when my server down for certain reason or my console disconnected (need time to reconnect), i assume that i will lose some tweets (i have 40 keywords and more). is this right?

i use filter-track.php for experiment

$sc = new FilterTrackConsumer(OAUTH_TOKEN, OAUTH_SECRET, Phirehose::METHOD_FILTER);
$sc->setTrack(array('plonknimbuzz'));
$sc->consume();

i also read

 /**
   * Sets the number of previous statuses to stream before transitioning to the live stream. Applies only to firehose
   * and filter + track methods. This is generally used internally and should not be needed by client applications.
   * Applies to: METHOD_FILTER, METHOD_FIREHOSE, METHOD_LINKS
   *
   * @param integer $count
   */
 public function setCount($count)
  {
    $this->count = $count;
  }

so i change filter-track.php to

$sc = new FilterTrackConsumer(OAUTH_TOKEN, OAUTH_SECRET, Phirehose::METHOD_FILTER);
$sc->setTrack(array('plonknimbuzz'));
$sc->setCount(100);
$sc->consume();

but i always face

Phirehose: HTTP failure 1 of 20 connecting to stream: HTTP ERROR 416: Requested
Range Not Satisfiable (Parameter count not allowed in role statusDefaultFiltered
). Sleeping for 10 seconds.

thanks for any explaination.

fennb commented

Hi @plonknimbuzz,

You're absolutely on the right track, the setCount parameter does do what you want. Unfortunately, they have restricted this functionality to twitter accounts with special privileges, see here: https://dev.twitter.com/streaming/overview/request-parameters#count

You would deep to apply to the Twitter team to be able to use this functionality.

The general approach they recommend is to use a mix of the REST and streaming API if you want to backfill past tweets.

@fennb your answer very usefull for me.
ok, i will mix REST API and STREAMING API. actually: i already doing it right now,

may i ask something more
the doc said:

When reconnecting to a streaming endpoint, elevated access clients may include the count parameter to attempt to backfill missed messages ....

and you said

You would deep to apply to the Twitter team to be able to use this functionality.

the meaning is: i only can used this count parameter if i become paid client or using gnip, isn't?

thanks for your great work. Awesome