Multiple processes accessing a shared resource concurrently
This example starts up a shared database and three different processes.
The processes represent multiple users, or locations, or programs hitting a shared database at the same time.
Complete the setup at streaming-01-getting-started.
Execute about.py to generate some useful information.
Executing multiple_processes.py script.
Read the output. Read the code. Try to figure out what's going on. #Ashley Allen
- What libraries did we import? #sqlite3, time, multiprocessing, os, datetime, platform, and sys
- Where do we set the task_duration? #Line 158
- How many functions are defined? #7
- What are the function names? #Create_table, drop_table, insert_pet, rocess_one, process_two, process_three, and recreate_database
- In general, what does each function do? #Create tables and insert the correct informtion.
- Where does the execution begin? #The execution begins on
- How many processes do we start? #3 Processes
- How many records does each process insert? #2 records
In this first run, we start 3 processes, each inserting 2 records into a shared database (for a total of 6 records inserted.)
In each case, the process gets a connection to the database, and a cursor to execute SQL statements. They insert a record, and get out of the database quickly.
In general, we're successful and six new records get inserted.
For the second run, modify the task_duration to make each task take 3 seconds. Run it again. With the longer tasks, we now get into trouble - one process will have the database open and be working on it - then when another process tries to do the same, it can't and we end up with an error.
To clear the terminal, in the terminal window, type clear and hit enter or return.
clear
To document results, clear the terminal, run the script, and paste all of the terminal contents into the output file.
Use out0.txt to document the first run.
Use out3.txt to document the second run.
On Windows the select all, copy, paste hotkeys are:
- CTRL a
- CTRL c
- CTRL v
On a Mac the select all, copy, paste hotkeys are:
- Command a
- Command c
- Command v
Detailed copy/paste instructions (as needed)
- To use these keys to transfer your output into a file, clear the terminal, run the script, then click in the terminal to make it active.
- To select all terminal content, hold CTRL and the 'a' key together.
- To copy the selected content, hold CTRL and the 'c' key together.
- To paste, open the destination file (e.g. out0.py) for editing.
- Click somewhere in the destination file to make it the active window.
- Now hit CTRL a (both together) to select all of the destination file.
- Hit CTRL v (both together) to paste the content from your clipboard.
Do a web search to find helpful videos on anything that seems confusing.
Python has pretty helpful error messages. When you get an error, read them carefully.
- What error do you get? Database is locked
- Can you tell what line it was executing when it failed? Yes
Do a web search on the sqlite3 'database is locked' error.
- What do you learn? #When two inapproopriate operations on a database.
- Once a process fails, it crashes the main process and everything stops.
Deadlock is a special kind of locking issue where a proces is waiting on a resource or process, that is waiting also.
Rather than crashing, a system in deadlock may wait indefinitely, with no process able to move forward and make progress.
Check out Wikipedia's article on deadlock and other sources to learn how to prevent and avoid locking issues in concurrent processes.