awslabs/mountpoint-s3

Limiting with the 「--maximum-troughput-gbps」 option does not work

mura890 opened this issue · 2 comments

Mountpoint for Amazon S3 version

mount-s3 version:1.7.2

AWS Region

ap-northeast-1

Describe the running environment

I am running an S3 bucket in the same account mounted on EC2 (OS: CentOS7.9)

Mountpoint options

mount-s3 test-S3-bucket --prefix archives/ /mnt/applications/classtream/archives --maximum-throughput-gbps 1 --uid 1006 --gid 1006 --file-mode 0660 --dir-mode 0770 --allow-delete --allow-other --allow-overwrite --cache /mnt/s3-cache --metadata-ttl 300

What happened?

Even if you add the "--maximum-troughput-gbps 1" option, the 1Gbps limit is not applied when copying files to S3.

I executed the copy command on five terminals as shown below to copy five 20GB files to the S3 area at the same time.
cp -p /mnt/20g.mp4 /mnt/applications/classtream/archives/aaa
cp -p /mnt/20g.mp4 /mnt/applications/classtream/archives/bbb
cp -p /mnt/20g.mp4 /mnt/applications/classtream/archives/ccc
cp -p /mnt/20g.mp4 /mnt/applications/classtream/archives/ddd
cp -p /mnt/20g.mp4 /mnt/applications/classtream/archives/eee

I monitored the network traffic with the "dstat -tams --disk-util" command and expected the send value to be 1gbps (125MB/s), but it was output as 400-500MB/s.
----system---- ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system-- ------memory-usage----->
time |usr sys idl wai hiq siq| read writ| recv send| in out | int csw | used buff cach free>
31-07 07:26:29| 7 7 74 11 0 2| 256M 0 |2726k 524M| 0 0 | 64k 36k|1987M 2088k 11.3G 2066M>
31-07 07:26:30| 6 6 76 10 0 1| 258M 0 |2754k 489M| 0 0 | 63k 33k|1986M 2088k 11.9G 1479M>
31-07 07:26:31| 5 6 79 9 0 1| 257M 18k|2443k 437M| 0 0 | 58k 30k|1983M 2088k 12.4G 937M>
31-07 07:26:32| 6 6 77 10 0 1| 256M 0 |2756k 456M| 0 0 | 59k 32k|1985M 2088k 12.9G 372M>
31-07 07:26:33| 7 7 76 8 0 2| 257M 2048B|2758k 502M| 0 0 | 62k 35k|1996M 2088k 13.1G 160M>
31-07 07:26:34| 5 6 79 9 0 1| 264M 0 |2369k 425M| 0 0 | 55k 30k|2000M 2088k 13.1G 160M>
31-07 07:26:35| 6 7 78 9 0 1| 257M 0 |2292k 449M| 0 0 | 54k 30k|2007M 2088k 13.1G 182M>
31-07 07:26:36| 6 7 76 9 0 1| 256M 0 |2337k 484M| 0 0 | 58k 33k|2007M 2088k 13.1G 170M>
31-07 07:26:37| 6 8 73 12 0 2| 257M 0 |2564k 517M| 0 0 | 62k 35k|2006M 2088k 13.1G 169M>

Based on this result, it was determined that there was no restriction due to "--maximum-troughput-gbps 1".
The instance type is c6in.2xlarge. Even if I set the "--max-threads" option to 1, the phenomenon did not change.

I don't understand why there is no limit.

Relevant log output

No response

arsh commented

We're now looking into this and will report back as soon as we have an update.

Thanks!