MIMS is intended to synchronize member accounts between National database and Wing accounts on the Google WorkSpace platform. MIMS provides the following functions: Add new members, remove members no longer found on National rolls, Suspend expired member accounts until they either fall off the rolls, or renew. MIMS scans the National database and emits batch jobs to perform it's various tasks. In addition to pure member account management MIMS also provides some automatically generated reports. Reports are provided as CSV formatted files for spreadsheet import. MIMS also provides a Group manager to manage both wing and unit level Google groups. MIMS is intended to be platform agnostic although it is designed primarily with Linux in mind.
MIMS uses Google custom schemas to more easily track member information like CAPID, Unit and member Type. This has been done as it is not possible to update a single attribute in the Google "organizations" schema, you must update the entire schema or lose data not supplied. Custom schemas allow individual attributes to be updated without disturbing existing information.
MIMS has been specifically designed as a batch only system. There are no fancy GUI's, no point-and-click web interfaces. It is bare bones grunt level batch processing. From the importing of data to the processing of jobs everything is controlled by a few scripts. The scripts have been designed to be used with a scheduler, i.e. cron or its equivalent, or directly from the command line. Once scheduled everything is automatic, although some jobs like purging members are put on hold and require an administrator to review them before releasing them. Jobs can be scheduled to run as frequently as you like. MIMS can be run on either a local dedicated server or a cloud instance. I personally use a Linux cloud instance reachable only via a secure and encrypted connection.
MongoDB JS scripts have been updated to work with the newer Mongo Shell (mongosh) 1.5>
- O/S: Linux, Windows 10>*
- Python 3.5>
- GAMADV-XTD3 Ross Scroggs excellent fork of the original GAM tool by Jay Lee.
- MongoDB document database manager 4.2>
- Mongo Shell (mongosh) 1.5>
- Google Workspace admin account.
- Cygwin environment
- Additional Cygwin tools: text-utils, bash, sed, tail, unzip
- iPython - interactive Python environment (makes life so much better for debugging)
- Studio 3T - MongoDB browser/editor
Note: Studio 3T is free for non-commercial use. I highly suggest purchasing Pro edition licenses if you are serious about development and maintenance. Pro licenses are available to CAP at the non-profit pricing level.
-
capwatch2.py - CAPWATCH downloader uses the new download API.
-
mims.py - MIMS batch job generator
-
jobexec - Job executive runs jobs created by MIMS (bash script)
-
getTables - Unpack CAPWATCH tables and filter for import into MongoDB (bash script)
-
holdMember - put member accounts on hold to prevent removal by MIMS, remove or list held accounts.
-
importGoogle - Google data downloader and mongoDB import, index collection script included (bash script)
-
importTables - Load CAPWATCH tables into mongoDB, index collections script included (bash script). Allows execution of pre & post MongoDB JS scripts on a per-collection basis. Script naming convention is TableName-[pre|post].js, e.g. "DutyPosition-pre.js". If a script is found it is automatically run, if no script is found no pre/post ops are preformed and the table is imported directly. Scripts must be located in the same directory as importTables.
-
jsonfix.sed - sed (Stream Editor) script used by importGoogle.
-
MIMSJobsGen - bash convinence script to generate GAM job files for a list of MIMS classes.
-
mongodb-seed - Mongo shell script to create empty database and base collections.
-
Note: bash script utilities have not been ported to the Windows environment yet. They work with cygwin bash shell.
Each component comes with a companion configuration file must be set prior to running MIMS. At minimum before you can run MIMS you must install and configure GAMADV-XTD3, Python3 and MongoDB. There is no installer for MIMS. Copy all of the MIMS components into a directory under your workspace, MIMS is a good name for it. Once installed you will need to update the configuation files, credential files and bash shell scripts with defaults appropriate to your installation. Config vars for the bash scripts are located .conf file for each main script, e.g. importGroups conf file is importGroups.conf. Python files will have a conf file and possibly a credentials file, e.g. capwatch2.py includes capwatch2_creds.py and capwatch2_conf.py. You will need to examine the mims_conf.py file and set each configuration variable as necessary.
MIMS uses capwatch2.py to download CAPWATCH data from eServices. The eServices release included a PowerShell implementation, capwatch2.py is a Python 3 rewrite and platform agnostic. NOTE: you will need to have your WSA and wing commander authorize your use of the api, and web interface.
- pymongo
Python modules may be installed using Python pip: pip install module_name
- Date-fns - JavaScript date calculations library.