A suite of tools to visualize pageviews data of Wikimedia Foundation wikis
Live tool: https://tools.wmflabs.org/pageviews (and Langviews, Topviews, Siteviews, Massviews, Redirect Views, Userviews, Mediaviews)
- User documentation: https://meta.wikimedia.org/wiki/Pageviews_Analysis
- JavaScript documentation: https://tools.wmflabs.org/pageviews/jsdocs
- Metaviews (pageviews of the Pageviews apps): https://tools.wmflabs.org/metaviews
- Toolforge maintainer documentation: https://wikitech.wikimedia.org/wiki/Tool:Pageviews
IE10 and Safari 8 and below are not supported.
The source code for Pageviews Analysis and it's sister applications is released under the MIT license.
The underlying data shown in these applications was provided by the Wikimedia RESTBase API, released under the CC0 1.0 license dedication.
Screenshots of the charts in Pageviews Analysis may be used without attribution, but a link back to the application would be appreciated.
This guide mostly assumes you're using MacOS or Linux. The setup process may differ on Windows machines.
-
PHP 5.6+ – PHP is for rendering partials in the views and for the Intuition i18n framework, and for some internal APIs that query the replicas.
-
ESLint and SCSS-lint – The linters are not necessary, but preferred so your code maintains a consistent style.
-
Install all node packages and dependencies with
npm install. -
Run
composer installto install all PHP dependencies. -
Create a copy of
config.sample.phpnamedconfig.php. See the comments within that file on what you need to configure. -
Optionally create the "meta" database on your local machine. This database is used for usage tracking, and also to store known false positives for Topviews. Unless you're working on Topviews, you can skip this step.
-
Run
gulpto watch the javascripts, stylesheets and views and automatically recompile when new changes are saved. -
Start the server with
php -S localhost:8000
You should be able to access each application as a subpath of localhost:8000, e.g. http://localhost:8000/pageviews.
This app aims to be a part of the future and not linger in the past. JavaScript is written in ES6 where possible, and transpiled to ES5 with Babel. If you need to add a polyfill for something, add it to /javascript/shared/polyfills.js.
All assets and views are ultimately placed in public_html. With the exception of images, you won't need to make any manual modifications to this directory.
The repo contains eight separate applications that share code with each other (Pageviews, Langviews, Topviews, Siteviews, Massviews, Redirect Views, Userviews, and Mediaviews). Each app has a dedicated subdirectory within javascripts, stylesheets and views. The main development asset files share the same name as the app (e.g. pageviews.js for the main JavaScript file for Pageviews). After compilation each app has it's own application.js and application.css in the public_html directory. PHP partials and Sass imports are prepended with underscores (e.g. _footer.php, _mixins.scss).
There are symlinks all throughout the public_html directory. These are to ensure you can use the native PHP server on your local machine and also deploy to Toolforge with the same code. The symlinks effectively change the document root of Toolforge's lighttpd server.
Browserify is used to help follow a module pattern. You'll need to require('./file_name') any file that is a dependency. All JavaScript is written in ES6 (and possibly ES7).
Each app has it's own config.js, which are constants for application-wide use. Pageviews, Siteviews and Mediaviews also have a templates.js file that defines how data is shown in the right column on the interface.
Shared JavaScript goes in the /shared directory and must be required as needed. list_helpers.js is used on apps that have a list view, which are Massviews, Langviews, Redirect Views and Userviews. All apps but Topviews show a chart of some sort, and require chart_helpers.js.
When the JS files are compiled, they are concatenated into a single application.js that lives within the directory for that app inside public_html.
All JavaScript is documented using JSDoc. The documentation is hosted at https://tools.wmflabs.org/pageviews/jsdocs. You can generate the docs locally by running:
gulp jsdoc && open jsdocs/gen/index.html
Styles are written in http://sass-lang.com/ and compiled to CSS. The CSS is then automatically vendor-prefixed to support modern browsers. Bootstrap is used as the CSS framework.
Each page has it's own .scss file that imports dependencies. Shared files are simply prefixed with an underscore. _mixins.scss is for the mixins, placeholders, keyframes and colour variables. Similar to the JavaScripts, _list_view.scss is imported by apps that have a list view (Massviews, Langviews, Redirect Views and Mediaviews), and _chart_view.scss by apps that show charts.
The views within /views are written in MtHaml and compiled to PHP files in /public_html.
Run gulp to watch for changes and automatically compile as needed. You can also run tasks by app and function, such as gulp massviews for all Massviews-related files, or gulp scripts to compile only the JavaScripts, but gulp by itself should be all you need.
Before making a pull request or pushing to master, remember to run gulp production so the assets are minified and concatenated. JSDocs are also generated, and placed in a dedicated sub-repo (not submodule) in jsdocs/gen. These can optionally be pull requested to the JSDocs repo.
For deployment instructions, see Tool:Pageviews on Wikitech.org.
Tests will be ran automatically when you push or create a pull request.
Pageviews Analysis was once tested with acceptance tests, but after significant maintenance burden and general flakiness of the tests, they have been retired :(
The styling of all ES6 and SCSS is enforced with linters. You can run these locally with gulp lint, and will also be ran when you run gulp production. If you need a particular rule to be ignored, you can add exceptions (see Scss-lint, Eslint).