Course Discovery Service

Service providing access to consolidated course and program metadata.

Documentation

Documentation is hosted on Read the Docs. The source is hosted in this repo's docs directory. The docs are automatically rebuilt and redeployed when commits are merged to master. To contribute, please open a PR against this repo.

License

The code in this repository is licensed under version 3 of the AGPL unless otherwise noted. Please see the LICENSE file for details.

How To Contribute

Contributions are welcome. Please read How To Contribute for details. Even though it was written with edx-platform in mind, these guidelines should be followed for Open edX code in general.

Development

Using elasticsearch locally

To use elasticsearch locally, and to update your index after adding new data that you want elasticsearch to access run:

$ ./manage.py update_index --disable-change-limit

To delete elasticsearch old indexes locally you have to use

$ ./manage.py remove_unused_indexes

This command will purge the oldest indexes, freeing up disk space. This command will never delete the currently used indexes.

Also you can use base commands by django-elasticsearch-dsl library.

Delete all the currently used indexes in Elasticsearch:

$ ./manage.py search_index --delete [-f] [--models [app[.model] app[.model] ...]]

Create the indices and their mapping in Elasticsearch:

$ ./manage.py search_index --create [--models [app[.model] app[.model] ...]]

Populate the Elasticsearch mappings with the django models data (index need to be existing):

$ ./manage.py search_index --populate [--models [app[.model] app[.model] ...]] [--parallel]

Recreate and repopulate the indices:

$ ./manage.py search_index --rebuild [-f] [--models [app[.model] app[.model] ...]] [--parallel]

Please use the link to get more https://django-elasticsearch-dsl.readthedocs.io/en/latest/management.html

WARNING: Be aware that search_index command works without sanity index check. So be careful to use it.

Working with memcached locally

Some endpoints, such as /api/v1/courses, have their responses cached in memcached through mechanisms such as the CompressedCacheResponseMixin. This caching may make it difficult to see code changes reflected in various endpoints without first clearing the cache or updating the cache keys. You can update the cache keys by going to any course_metadata model in the admin dashboard and clicking save. To flush your local memcached, make sure the edx.devstack.memcached container is up and run:

$ telnet localhost 11211
$ flush_all
$ quit

Running Tests Locally, Fast

There is a test settings file course_discovery.settings.test_local that allows you to persist the test database between runs of the unittests (as long as you don't restart your container). It stores the SQLite database file at /dev/shm, which is a filesystem backed by RAM. Using this test file in conjunction with pytest's --reuse-db option can significantly cut down on local testing iteration time. You can use this as follows: pytest course_discovery/apps/course_metadata/tests/test_utils.py --ds=course_discovery.settings.test_local --reuse-db

The first run will incur the normal cost of database creation (typically around 30 seconds), but the second run will completely skip that startup cost, since the --reuse-db option causes pytest to use the already persisted database in the /dev/shm directory. If you need to change models or create databases between runs, you can tell pytest to recreate the database with -recreate-db.

Debugging Tests Locally

Pytest in this repository uses the pytest-xdist package for distributed testing. This is configured in the pytest.ini file. However, pytest-xdist does not support pdb.set_trace(). In order to use pdb when debugging Python unit tests, you can use the pytest-no-xdist.ini file instead. Use the -c option to the pytest command to specify which ini file to use.

For example,

pytest -c pytest-no-xdist.ini --ds=course_discovery.settings.test --durations=25 course_discovery/apps/publisher/tests/test_views.py::CourseRunDetailTests::test_detail_page_with_comments

Reporting Security Issues

Please do not report security issues in public. Please email security@edx.org.

Get Help

Ask questions and discuss this project on Slack or in the edx-code Google Group.

johnnagro/course-discovery