twitter4s

An asynchronous non-blocking Scala Twitter Client, implemented using Akka-Http and json4s.

Prerequisites

JDK 8
Scala 2.12+

Go to https://developer.twitter.com/, login with your twitter account and register your application to get a consumer key and a consumer secret.
Once the app has been created, generate a access key and access secret with the desired permission level.

NOTE: v1.1 endpoints are unaccessible for new users. More information at https://twittercommunity.com/t/ushering-in-a-new-era-for-the-twitter-developer-platform-with-the-twitter-api-v2/162087

Rate Limits

Be aware that the Twitter REST Api has rate limits specific to each endpoint. For more information, please have a look at the Twitter developers website here.

For all the endpoints that are affected these limitations, information on the current rates together with the requested data is provided by the RatedData case class.

Setup

If you don't have it already, make sure you add the Maven Central as resolver in your SBT settings:

resolvers += Resolver.sonatypeRepo("releases")

Also, you need to include the library as your dependency:

libraryDependencies += "com.danielasfregola" %% "twitter4s" % "8.0"

Giter8

If you are starting from scratch, you can use twitter4s.g8 template to start your project. This template contains examples for both REST and Streaming client.

> sbt new DanielaSfregola/twitter4s.g8

Usage

Add your consumer and access token as either environment variables or as part of your configuration. Twitter4s will look for the following environment variables:

export TWITTER_CONSUMER_TOKEN_KEY='my-consumer-key'
export TWITTER_CONSUMER_TOKEN_SECRET='my-consumer-secret'
export TWITTER_ACCESS_TOKEN_KEY='my-access-key'
export TWITTER_ACCESS_TOKEN_SECRET='my-access-secret'

You can also add them to your configuration file, usually called application.conf:

twitter {
  consumer {
    key = "my-consumer-key"
    secret = "my-consumer-secret"
  }
  access {
    key = "my-access-key"
    secret = "my-access-secret"
  }
}

These configurations will be automatically loaded when creating a twitter client, so all you have to do is to initialize your clients as following:

import com.danielasfregola.twitter4s.TwitterRestClient
import com.danielasfregola.twitter4s.TwitterStreamingClient

val restClient = TwitterRestClient()
val streamingClient = TwitterStreamingClient()

Alternatively, you can also specify your tokens directly when creating the client:

import com.danielasfregola.twitter4s.TwitterRestClient
import com.danielasfregola.twitter4s.TwitterStreamingClient
import com.danielasfregola.twitter4s.entities.{AccessToken, ConsumerToken}

val consumerToken = ConsumerToken(key = "my-consumer-key", secret = "my-consumer-secret")
val accessToken = AccessToken(key = "my-access-key", secret = "my-access-secret")  

val restClient = TwitterRestClient(consumerToken, accessToken)
val streamingClient = TwitterStreamingClient(consumerToken, accessToken)

Once you have instantiated your client you are ready to use it! 😄

Twitter Streaming Client

TwitterStreamingClient is the client to support stream connections offered by the Twitter Streaming Api.

You can initialize the client as follows:

import com.danielasfregola.twitter4s.TwitterStreamingClient

val client = TwitterStreamingClient()

There are three types of streams, each with different streaming message types: Public Stream, User Stream, Site Stream.

Each stream requires a partial function that indicates how to process messages. If a message type is not specified, it is ignored. See the section of each stream for more information.

For example, you can create the following function to print the text of a tweet:

import com.danielasfregola.twitter4s.entities.Tweet
import com.danielasfregola.twitter4s.entities.streaming.StreamingMessage

def printTweetText: PartialFunction[StreamingMessage, Unit] = {
    case tweet: Tweet => println(tweet.text)
}

All you need to do is attach your processing function to the stream:

client.sampleStatuses(stall_warnings = true)(printTweetText)

...and you are done, happy days! 👯

Have a look at TwitterProcessor for some predefined processing functions.

Close or Replace a Stream

Each stream function returns a Future[TwitterStream]. TwitterStream represents the stream received by Twitter and it can be used to close or replace the current stream.

For example, consider the following snippet:

  // TERRIBLE CODE! NEVER BLOCK! Code for demo purposes only!
  def simulateNextActionAfterMillis(millis: Long): Future[Unit] = Future{ Thread.sleep(millis); println() }

  for {
    streamA <- client.sampleStatuses(languages = Seq(Language.English)){ case t: Tweet => print("o")}
    _ <- simulateNextActionAfterMillis(10000)
    streamB <- streamA.sampleStatuses(languages = Seq(Language.Spanish)){ case t: Tweet => print("+")}
    _ <- simulateNextActionAfterMillis(10000)
  } yield streamB.close()

The above code can output something similar to the following:

oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
+++++++++++++++

In this example, we can see that there are more English tweets than Spanish tweets.

Public Stream

Have a look at the complete scaladoc for the Public Stream Client.

Available streams

filterStatusesFilter
sampleStatusesSample
firehoseStatuses

CommonStreamingMessage types:

User Stream

Have a look at the complete scaladoc for the User Stream Client.

Available streams

userEvents

UserStreamingMessage types:

All the instances of CommonStreamingMessage -- see the Public Stream Section
FriendsLists and FriendsListsStringified
SimpleEvent
TweetEvent
TwitterListEvent

Site Stream

Have a look at the complete scaladoc for the Site Stream Client.

Available streams

siteEvents

SiteStreamingMessage types:

All the CommonStreamingMessages -- see the Public Stream Section
ControlMessage
UserEnvelopTweet and UserEnvelopTweetStringified
UserEnvelopDirectMessage and UserEnvelopDirectMessageStringified
UserEnvelopSimpleEvent and UserEnvelopSimpleEventStringified
UserEnvelopTweetEvent and UserEnvelopTweetEventStringified
UserEnvelopTwitterListEvent and UserEnvelopTwitterListEventStringified
UserEnvelopFriendsLists and UserEnvelopFriendsListsEventStringified
UserEnvelopWarningMessage and UserEnvelopWarningMessageStringified

Documentation

The complete scaladoc with all the available streams for the TwitterStreamingClient can be found here.

Twitter REST Client

TwitterRestClient is the client for the REST endpoints offered by the Twitter REST Api.

Once you have configured your consumer and access token, you can initialize an instance of TwitterRestClient as follows:

import com.danielasfregola.twitter4s.TwitterRestClient

val client = TwitterRestClient()

For example, you can get the home timeline of the authenticated user:

client.homeTimeline()

or you can get the timeline of a specific user:

client.userTimelineForUser("DanielaSfregola")

You can also update your tweet status:

client.tweet(status = "Test")

Asynchronous upload of images or short videos is also supported:

for {
  upload <- client.uploadMediaFromPath("/path/to/file.png")
  tweet <- client.tweet(status = "Test with media", media_ids = Seq(upload.media_id))
} yield tweet

Documentation

The complete scaladoc with all the available functionalities for the TwitterRestClient can be found here.

TwitterRestClient is composed by several traits. A list of the supported resources is following:

Proxy Support

If needed, you can redefine the domain used for each of the twitter api by overriding the following settings in your configuration file:

twitter {

  rest {
    api = "https://api.twitter.com"
    media = "https://upload.twitter.com"
  }

  streaming {
    public = "https://stream.twitter.com"
    user = "https://userstream.twitter.com"
    site = "https://sitestream.twitter.com"
  }
}

Logging

Twitter4s uses scala-logging and can be used in your twitter4s application.

In your application you will need a logging backend (logback, logstash). logback-classic is easy to use and will suit most needs. You can find a sample configuration in twitter4s-demo and in the Giter8 template twitter4s.g8

Examples

Have a look at the repository twitter4s-demo for more examples on how to use twitter4s.

Snapshot Versions

To use a snapshot version of this library, make sure you have the resolver for maven central (snapshot repositories) in your SBT settings:

resolvers += Resolver.sonatypeRepo("snapshots")

Then, add the library as your dependency:

libraryDependencies += "com.danielasfregola" %% "twitter4s" % "8.1-SNAPSHOT"

Coming up Features

OAuth1 support
Advanced query support
Support for dump to file
...

How to Contribute

Contributions and feature requests are always welcome!

Fork the repo and checkout the code
Make sure to run sbt with jdk8+
Run the tests with sbt test
...you can now do your magic and submit a PR when you are ready! 👯