We have converted the 5000 page Ferguson Grand Jury transcript PDF into something more readable. This conversion makes the transcript searchable and linkable. We hope these features add to the national discussion about institutional racism and police brutality. We're using the SayIt service by MySociety.
The Ferguson Grand Just testimony PDF is 5000 pages long. It was transcribed over 100 days by different people, so the formatting is all over the place. There is also a lot of redacted content. We've done the best we can to get these transcripts up and live. Done is better than perfect. Compare anything that looks weird to the official released PDF.
- Share with your local media
- Find and share the most important parts of the transcripts.
- Review the transcripts and write down any issues.
- Improve our converter by writing code to fixing issues
- OCR the transcripts
- Use
pdf2txt.pyto convert PDF to XML - Use
parse_transcript_xml.pyto convert XML to formatted txt - Use
converted_text_to_akoma_ntoso.pyscript to convert the new formatted text to Akoma Ntoso - Upload to SayIt
- Promote to media and activists
http://graphics8.nytimes.com/newsgraphics/2014/11/24/ferguson-assets/grand-jury-testimony.pdf
https://www.dropbox.com/s/67unqhdrb8jhgr0/Ferguson%20Grand%20Jury%20Testimony.pdf?dl=0
parse_transcript_xml.py is a script that parses the XML files created by running pdf2txt.py (PDFMiner) on the OCR Transcripts.
The XML files generated by PDFMiner preserve formatting information which we can use to more accurately identify text attributed to individuals whose names have been redacted from the transcript, by examining text indentations.
The XML file of the full transcript can be generated with the command:
pdf2txt.py -o files/ferguson_grand_jury_testimony.xml Ferguson\ Grand\ Jury\ Testimony.pdf