It is a platform mainly used for performance testing, here are some simple features brief:
- Server Management, can view server's basic information uniformly
- Shell Remote Connection, support for files upload and download between local and server
- Server resource usage monitoring
- Nginx's access.log traffic collection
- Performance Testing tool, support for automated and distributed performance testing
- MyPlatform - project files
- staticfiles - static files
- templates - html templates files
- templateFilter - custom filter
- common - generic functions
- user - user related
- shell - shell tool
- monitor - monitor tool
- performance - performance test tool
- Relational Database: SQLite3 or MySQL - used to store platform data
- Time-Series Database: InfluxDB - used to store monitoring data
- Key-value Database: Redis - used to cluster/distributed data synchronization
- File Server: MinIO - used to store files
- Performance Testing tool: JMeter - used to execute JMeter file
If you need to satisfy more users, please deploy cluster; if you need high availability, please deploy keepalive.
collector-agent
Data Collector. All agents' data will be sent to collector-agent, and then collector-agent writes data to InfluxDB/redis.
It can be avoid a problem: If each agent connects to the database separately, it may cause the database connection to run out or exceed the number of connections allowed by the server.
But if too many agents cause the collector-agent to not be able to write database in time, increasing the thread pool size of the collector-agent is needed; if still not, increasing the number of collector-agent cluster nodes is needed.
monitor-agent
Server resource monitor. Execute Linux commands to collect the server's CPU, Memory, Disk, Network, TCP, and other data in real time.
nginx-agent
Nginx traffic collector. Process Nginx's access log (access.log) in real time, the access information (access time, client IP, interface name, request method, protocol, status code, response body size, response time) is stored in database.
jmeter-agent
Performance testing tool. Call JMeter to execute performance testing, and supports distributed performance testing and full-link performance testing.
-
Clone Repository
git clone https://github.com/leeyoshinari/MyPlatform.git
-
Install MySQL(SQLite3 can be used directly, doesn't need to be installed), InfluxDB, Redis, MinIO(Optional installation); (ps:InfluxDB2.x is not supported, influxdb-1.8.3 is recommended.)
-
Install third-party packages
pip3 install -r requirements.txt
-
Modify
config.conf
; -
Initialize database, and execute commands
python3 manage.py migrate python3 manage.py makemigrations shell performance python3 manage.py migrate
-
Create super administrator
python3 manage.py createsuperuser
-
Initialize data
python3 manage.py loaddata initdata.json
-
Collect all static files
python3 manage.py collectstatic --clear --noinput
-
Compress static files (css and js)
python3 manage.py compress --force
-
Modify Port in
startup.sh
-
Deploy
nginx
, thelocation
configuration is as follows: (ps: Theplatform
in the configuration is the prefix, that is the URL prefix in the URL path, which can be modified according to your needs.)
(1) upstream configuration:upstream myplatform-server { server 127.0.0.1:15200; server 127.0.0.1:15201; }
(2) static request: Use Nginx to access static files directly
location /platform/static { alias /home/MyPlatform/static; }
(3) dynamic request:
location /platform { proxy_pass http://myplatform-server; proxy_set_header Host $proxy_host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; }
(4) websocket protocol:
location /shell { # must be shell, don't modify it proxy_pass http://myplatform-server; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; }
-
Startup
sh startup.sh
Run
sh shutdown.sh
to stop. -
Access home page, url:
http://ip:port/(prefix in config.conf)
-
Access permission management page, url:
http://ip:port/(prefix in config.conf)/admin
-
Deploy collector-agent, please click me
-
Deploy monitor-agent, please click me
-
Deploy jmeter-agent, please click me
-
Deploy nginx-agent, please click me
- For more information, please click me or view course after deployment.
Local dev environment:
- python 3.9.10
Third-party packages version:
- aiohttp==3.7.4.post0
- aiohttp-jinja2==1.5
- channels==3.0.4
- daphne==3.0.2
- Django==4.0.1
- django-compressor==4.1
- influxdb==2.6.0
- Jinja2==3.0.3
- minio==7.1.3
- paramiko==2.10.3
- PyMySQL==1.0.2
- redis==4.1.1
- requests==2.27.1
- sqlparse==0.4.2