Notes made while learning Python/Django
Installed python 3.6 and python package manager, 'pip'
Create a virtual environment:
python -m venv .virtualenvs/virtual_env_name
Activate virtual environment:
source .virtualenvs/virtual_env_name/bin/activate
Install Django:
pip install django
Initialise Django project, and create app:
django-admin startproject project_name
cd project_name
python manage.py startapp app_name
Add app to Project settings.py
settings.py
INSTALLED_APPS = (
...
'myapp',
...
)
To set Django which settings file Django will use:
export DJANGO_SETTINGS_MODULE=mysite.settings
(By default this will be automatically set)
A collection of useful command line extensions
pip install django-extensions
Add to Project settings.py:
INSTALLED_APPS = (
...
'django_extensions',
...
)
python manage.py shell_plus
Launches a Python shell with all your django models imported
python manage.py runscript <script>
Runs a Python script within your django environment. Create a folder, scripts, in your app.
Create your script eg. myscript.py within this folder.
def run():
# do something here...
Run with python manage.py runscript myscript --traceback
AWS S3 Bucket Policy:
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "PublicReadForGetBucketObjects",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::myBucket/*"
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::830686100485:user/DjangoS3Storages"
},
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::myBucket",
"arn:aws:s3:::myBucket/*"
]
}
]
}
pip install django-storages
pip install boto3
Add django-storages and AWS settings to Project settings.py:
settings.py
INSTALLED_APPS = [
...
'storages',
...
]
...
STATIC_ROOT = 'static'
AWS_STORAGE_BUCKET_NAME = 'bucket_name'
AWS_ACCESS_KEY_ID = 'aws_access_key'
AWS_SECRET_ACCESS_KEY = 'aws_secret_access_key'
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage'
STATICFILES_LOCATION = 'static'
STATIC_URL = '/static/'
DEFAULT_FILE_STORAGE = 'asset_manager.custom_storages.MediaStorage'
MEDIAFILES_LOCATION = 'media'
MEDIA_URL = "https://%s/%s/" % (AWS_S3_CUSTOM_DOMAIN, MEDIAFILES_LOCATION)
To allow media and static files to be stored separately, and use AWS S3, create `myapp/custom_storages.py':
from django.conf import settings
from storages.backends.s3boto3 import S3Boto3Storage
class StaticStorage(S3Boto3Storage):
location = settings.STATICFILES_LOCATION
class MediaStorage(S3Boto3Storage):
location = settings.MEDIAFILES_LOCATION
Test Static Files Collection:
python manage.py collectstatic
Create model classes in models.py Add to DB:
python manage.py makemigrations
python manage.py migrate
-
python manage.py createsuperuser
-
Register your models in
admin.py:admin.site.register(MyModel) -
Run Django's simple dev server:
python manage.py runserver -
Login at http://localhost:8000/admin
Used to monitor model pre-save, post-save etc functions, and run some code ie update AWS S3 when a Folder model is renamed
myproject.settings.py:
INSTALLED APPS=[
...
replace 'myapp' with
'myapp.apps.MyappConfig',
...
]
myapp.apps.py:
from django.apps import AppConfig
class MyappConfig(AppConfig):
name = 'myapp'
def ready(self):
import myapp.signals
myapp.signals.py
from django.db.models.signals import pre_save
from django.dispatch import receiver
from .models import MyModel
@receiver(pre_save, sender=MyModel)
def intercept_pre_save(sender, instance, **kwargs):
print('pre_save signal intercepted')
A useful utility package. FieldTracker is particularly helpful for tracking field changes
pip install django-model-utils
from model_utils import FieldTracker
Note that FieldTrackers track Foreign Key fields by db_column name, not model field name.
To find the db_column name of your fields:
>>> for field in MyModel._meta.fields:
field.get_attname_column
('id', 'id')
('name', 'name')
('parent_id', 'parent_id')
Delete tests.py from your app folder.
Replace with a tests folder, and add an empty __init__.py
Create a test file of the form test*.py:
from django.test import TestCase
class ModelTest(TestCase):
def setUp(self):
# code to run before each test
def tearDown(self):
# code to run after each test
def my_test(self):
"""
test description
"""
# test code here ...
self.assertEqual(a, b)
Run tests:
python manage.py test myapp
Run a specific test:
python manage.py test myapp.tests.test_file.test_class.test_function
To test code coverage
pip install coverage
coverage run --source='.' manage.py test
coverage report -m
ALSO
covergae html
This produces an html report at project/htmlcov/index.html
You can use a .coveragerc file, in the same directory as manage.py to specify options:
[run]
branch=true
source=asset_manager
hive
omit=asset_manager/migrations/*
asset_manager/tests.py
hive/wsgi.py
*/__init__.py
manage.py
see https://coverage.readthedocs.io/en/coverage-4.4.1/config.html
Then you can run: coverage run manage.py test
To run tests with a different settings file:
coverage run manage.py test --settings=project.settings-test
To run just a single test file eg. model_tests.py
coverage run manage.py test myapp.tests.model_tests
To disable signals during certain tests:
signals.post_save.disconnect(save_folder_to_s3, sender=Folder)
signals.post_delete.disconnect(delete_folder, sender=Folder)
Note TestCase methods are run alphabetically, independently of their TestCase class! This means that each method needs to be atomic, and must undo any changes that it has made. So signals will need to be reconnected if a method has disconnected them, after it has finished.
signals.post_save.disconnect(save_folder_to_s3, sender=Folder)
signals.post_delete.disconnect(delete_folder, sender=Folder)
setings.py:
TIME_ZONE = 'Europe/London'
LOFILE = 'logs/project-logs.txt'
from django.conf import settings
import logging
logging.basicConfig(
filename=settings.LOGFILE,
level=logging.INFO,
format=' %(asctime)s - %(levelname)s - %(message)s'
)
# logging.disable(logging.CRITICAL)
logging.debug('Start of program')
To get contents of AWS S3 Bucket programmatically (objects returned alphabetically)
s3 = boto3.client(
's3',
aws_access_key_id = settings.AWS_ACCESS_KEY_ID,
aws_secret_access_key = settings.AWS_SECRET_ACCESS_KEY)
bucket = settings.AWS_STORAGE_BUCKET_NAME
bucket_contents = []
for obj in s3.list_objects(Bucket = s3_utils.bucket)['Contents']:
bucket_contents.append(obj['Key'])
To delete programmatically
contents = s3.list_objects(Bucket = s3_utils.bucket)
if 'Contents' in contents:
for obj in contents['Contents']:
s3_utils.s3.delete_object(Bucket = s3_utils.bucket, Key = obj['Key'])
pip install psycopg2
You will need to install Postgres on your machine, then:
psql -U postgres
CREATE DATABASE 'db';
CREATE USER 'user' WITH PASSWORD 'password';
GRANT ALL PRIVILEGES ON DATABASE db TO user;
ALTER USER user CREATEDB;
settings.py:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'db',
'USER': 'user',
'PASSWORD': 'password',
'HOST': '',
'PORT': '',
}
}
\l list databases
\dt list tables in database
\q quit
\du list users
\connect DBNAME connect to a database
\conninfo Connection info
DROP DATABASE database_name; To delete a database (not the one currently connected)
DELETE FROM django_migrations WHERE app='your-app-name'; Delete all migrations
- Delete everything in your app/migrations folder (except init.py)
- DROP, CREATE and GRANT PRIVILEGES TO DB
- python manage.py makemigrations, migrate
see https://simpleisbetterthancomplex.com/tutorial/2016/07/26/how-to-reset-migrations.html
Check out these links:
https://www.youtube.com/watch?v=XphJRQ3AzMU
https://medium.com/@hakibenita/how-to-turn-django-admin-into-a-lightweight-dashboard-a0e0bbf609ad
https://medium.com/@hakibenita/how-to-add-custom-action-buttons-to-django-admin-8d266f5b0d41
Check out this great tutorial:
http://polyglot.ninja/django-building-rest-apis/
pip install djangorestframework
INSTALLED_APPS = [
...
rest_framework
...
]
models.py:
class Subscriber(models.Model):
name = models.CharField("Name", max_length=50)
age = models.IntegerField("Age")
email = models.EmailField("Email")
serializers.py:
from rest_framework import serializers
from . import models
class SubscriberSerializer(serializers.ModelSerializer):
class Meta:
model = models.Subscriber
fields = '__all__'
views.py:
from rest_framework.viewsets import ModelViewSet
from .serializers import SubscriberSerializer
from .models import Subscriber
class SubscriberViewSet(ModelViewSet):
serializer_class = SubscriberSerializer
queryset = Subscriber.objects.all()
urls.py:
from rest_framework.routers import SimpleRouter
router = SimpleRouter()
router.register("subscribers", SubscriberViewSet)
urlpatterns = router.urls
Create a user programmatically:
>>> from django.contrib.auth.models import User
>>> user=User.objects.create_user('foo', password='bar')
>>> user.is_superuser=True
>>> user.is_staff=True
>>> user.save()
Settings.py:
INSTALLED_APPS = [
...
rest_framework.authtoken
...
]
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': (
'rest_framework.authentication.BasicAuthentication',
'rest_framework.authentication.SessionAuthentication',
'rest_framework.authentication.TokenAuthentication',
)
}
Permissions: (views.py)
from rest_framework.permissions import IsAuthenticated
class SubscriberViewSet(ModelViewSet):
serializer_class = SubscriberSerializer
queryset = Subscriber.objects.all()
permission_classes = (IsAuthenticated,)
JSON Web Tokens:
pip install djangorestframework-jwt
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': (
...
'rest_framework_jwt.authentication.JSONWebTokenAuthentication',
...
)
}
urls.py:
from rest_framework_jwt.views import obtain_jwt_token
urlpatterns = router.urls + [
url(r'^jwt-auth/', obtain_jwt_token),
]
Obtain a token:
curl --request POST \
--url http://localhost:8000/api/jwt-auth/ \
--header 'content-type: application/json' \
--data '{"username": "user", "password": "pw"}'
{"token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjoyLCJlbWFpbCI6IiIsInVzZXJuYW1lIjoidGVzdF91c2VyIiwiZXhwIjoxNDk1OTkyOTg2fQ.sWSzdiBNNcXDqhcdcjWKjwpPsVV7tCIie-uit_Yz7W0"}
Use token to access collection:
curl -H "Content-Type: application/json" -H "Authorization: JWT eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjoyLCJlbWFpbCI6IiIsInVzZXJuYW1lIjoidGVzdF91c2VyIiwiZXhwIjoxNDk1OTkyOTg2fQ.sWSzdiBNNcXDqhcdcjWKjwpPsVV7tCIie-uit_Yz7W0" -X GET http://localhost:8000/api/subscribers/
Auto-setting uploaded_by and uploaded_at fields
settings.py:
TIME_ZONE = 'Europe/London'
USE_TC = True
admin.py:
MyModelAdmin(admin.ModelAdmin):
from django.conf import settings
from datetime import datetime
from pytz import timezone
def save_model(self, request, obj, form, change):
if not change:
obj.uploaded_by = request.user
obj.uploaded_at = datetime.now(timezone(settings.TIME_ZONE))
super(MyModelAdmin, self).save_model(request, obj, form, change)
Django admin templates can be found here: https://github.com/django/django/tree/master/django/contrib/admin/templates/admin
So far I have used list_view, which shows the list of all models of a given type, and change_view, which is the edit fields view for a given model.
settings.py:
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [os.path.join(BASE_DIR, 'templates')],
'APP_DIRS': True,
...
},
]
Create a templates folder within your app. Then, to override templates on an model specific basis, create a folder for each model:
myapp/templates/admin/myapp/mymodel
Put a copy of the required template in this folder, and edit as required. This local copy will now take precedence.
Note it is also possible to extend a template rather than copy it, and override specific blocks.
Create a directory for custom filters:
myapp/templatetags/
Create a file to hold your app filters:
myapp_filters.py:
from django import template
register = template.Library()
@register.filter
def show_root_level_folders_only(cl):
"""
cl is a variable holding a Change List; the list of model instances.
This function filters the Change List to remove non-root elements.
"""
query_set = cl.result_list
for folder in query_set:
if folder.parent != None:
query_set = query_set.exclude(pk=folder.id)
cl.result_list = query_set
return cl
Use the filter in your template. In this example, to remove non-root elements from a list, cl:
change_list.html:
{{ cl|show_root_level_folders_only }}
Django admin widgets are located at: https://github.com/django/django/blob/master/django/forms/widgets.py
The widget templates are in: django/contrib/admin/templates/admin/widgets/
settings.py:
INSTALLED_APPS = [
...
'django.forms',
...
]
FORM_RENDERER = 'django.forms.renderers.TemplatesSetting'
myapp/widgets.py:
from django import forms
class MyCustomWidget(forms.widgets.ClearableFileInput):
template_name = 'widgets/my_custom_widget.html'
myapp/forms.py:
class MyModelForm(forms.ModelForm):
class Meta:
model = MyModel
widgets = {
'file': widgets.MyCustomWidget,
}
Create template folder and template:
myapp/templates/widgets/my_custom_widget.html