Home
This is the wiki for PageOneX. We'll use this to document everything related to the project.
Help us document the project!
We have divided this wiki in two sections:
To start coding you must:
- Sign up in PageOneX.com
- Create a thread
- title
- description
- period: start and end date (we suggest to start short, 1 week, and then edit and make the thread larger)
- select newspapers
- topics (themes you want to code) and related color and description [you can edit and add more later]
- Code, that means drawing areas on top of the front page images.
- Display. PageOneX will display the data visualization with all the front pages and drawn areas and the bar chart on top of it.
From the display view it is possible to export to a multiple spreadsheet .ods file. Each of the sheets in the exported document contains the coverage per day per topic per newspaper. The first column of every sheet is a number. If you format the cell to "date format" it will render the date in human readable format.
From the display view it is possible to export to a jpg file. First you have to select the numbers of pixels you want for the width of your final image. To start with, 1200px is a good size, so just type "1200" without the ". You have two options:
- .jpg, downloads an image (it is crashing for big images, so better use the .zip option)
- .zip, downloads a zip folder with many files: two images per topic (just the areas and the areas an the front pages) + the composite image + just the newspaper images without the highlighted areas.
- ← = previous image
- → = next image
- n = nothing to code
Tutorial in slides in Spanish about How to use PageOneX by Anibal Rossi (in Spanish).
Video tutorial: How to use PageOneX (Spanish) | Cómo funciona PageOneX.
Video tutorial: How to create a thread in PageOneX | Cómo crear un hilo en PageOneX.
See more updated information about the data model: https://docs.google.com/presentation/d/1SnppQJLucy4OuTZ19oK94Q6S5tZY79TLEUvh3ljQlYs/edit#slide=id.ge24130b_0_0
How to deploy PageOneX locally or in your own server.
Note: You will need to have postgresSQL installed for the development environment, and mySQL for production environment.
You can avoid production environment by bundle --without production
-
Manage ruby versions via RVM (ruby version manager): https://rvm.io/rvm/install/
-
Use RVM to get ruby 1.9 and make it the default with RVM (https://rvm.io/rubies/installing/):
rvm install 1.9.3
Then force to use version 1.9.3:
rvm use 1.9.3
- Install bundler for good package management (http://gembundler.com):
gem install bundler
- Grab the PageOneX code from github: https://github.com/numeroteca/PageOneX
git clone [email protected]:numeroteca/pageonex.git
It will generate a directory called 'pageonex'.
- Go to the directory you cloned the code into
cd pageonex, and install all the gem dependencies listed in the Gemfile:
Install bundler:
gem install bundler
then
bundle install
- Set up the development sqlite database:
rake db:migrate
-
Load the media (newspaper data) to the database:
rake scraping:update_media -
Run the server:
rails server
You should be able to hit http://localhost:3000 and see something.
You will need to add to these gems to the Gemfile:
gem 'rails_12factor' # Heroku logging y servir assets (without this it doesn't serve images, css…)
gem 'thin'
gem 'nokogiri' Then you need to bundle install and push again to Heroku.
apt-get install ruby
apt-get install rubygems
apt-get install rails
apt-get install imagemagick
apt get install libmagick-dev
apt-get install libmagickwand-dev
apt-get install libxslt-dev libxml2-dev
apt-get install libpq-de
mysql2
curl -L https://get.rvm.io | bash -s stable
gem install bundlerChange the value of self.per_page = 20 in app/models/threadx.rb
The pagination is made using will_paginate gem and the styling with 'bootstrap-will_paginate'.
The default order of the threads is set at default_scope order('created_at DESC') in app/models/threadx.rb
First you need to run the scraper, it will check all the newspapers available in kiosko.net and will write the new found ones in public/kiosko_media_list.csv
rake scraping:update_media['public/kiosko_media_list.csv']
Then you need to update the list of newspapers into the data base:
rake scraping:update_media
Change value to false at config/initializers/application.rb:
Pageonex::Application.config.use_local_images = true
For Heroku: This is useful when the sever where the PageOneX is deployed has not much space. It will embed the images from kiosko.net
From the console (replace [email protected] with the user's email). The console should be run in production mode if you are using a production environment:
RAILS_ENV="production" rails console
or in development:
RAILS_ENV="development" rails console
Once inside the console, run the following:
User.find_by_email('[email protected]').update_attribute :admin, true
To add invalid usernames, just modify the list in the ok_username method of app/models/user.rb.
If you want to try and get all the images for a thread again:
Threadx.find_by_thread_name([slug]).scrape_all_images true
You can edit it in line 141 of the threadx model
Open rails console:
rails console production
And once inside:
Threadx.find_by_thread_name([slug]).scrape_all_images true