Skip to content

A web application for real-time machine learning and sentiment analysis on Tweets

License

Notifications You must be signed in to change notification settings

crawles/twitter-nlp

Repository files navigation

Twitter NLP

Twitter NLP is a microservice-based web application for analyzing Twitter sentiment in real time. The goal of the project is to show how to deploy a machine learning model, apply it in real time, and scale the model.

Check out this blog post and this blog post for more information.

Try the application: Twitter NLP

Resources:

Architecture

Alt text

Dashboard information:

  • Server Side: Python + Flask + SSE
  • Client Side: D3 for visualization and Server Sent Events (SSE) for real-time data streaming

Microservices

  • compute-tweet-stats: flask app for computing sentiment analysis stats (tweets/second and avg. sentiment)
  • firehose: connects to Twitter firehose and publishes Tweets to redis for the dashboard
  • gen-tweet-stats: gets the performance statistics generated from "compute-tweet-stats"
  • load-test-twitter-nlp: a load testing application for scale testing
  • sentiment-compute-app: sentiment analysis model accessible via a RESTful API
  • twitter-nlp: the dashboard

Deploying the app on Pivotal Cloud Foundry

Step 1

  • Rename manifest.example.yml to manifest.yml
  • In manifest.yml replace both *URL for compute-tweet-stats* and *URL for sentiment-compute-app* with appropriate URLs

Step 2

  • Add your Twitter credentials to CF environmental variables
cf set-env APP_NAME ACCESS_TOKEN ENV_VAR_VALUE
cf set-env firehose ACCESS_TOKEN_SECRET ENV_VAR_VALUE
cf set-env firehose CONSUMER_KEY ENV_VAR_VALUE
cf set-env firehose CONSUMER_SECRET ENV_VAR_VALUE

Step 3

cf create-service p-redis shared-vm twitter-nlp-redis
cf push

Notes

The project was inspired in part by BirdWatch

About

A web application for real-time machine learning and sentiment analysis on Tweets

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published