megvii-research/megfile

smart_open 不兼容 open pipe 管道

Closed this issue · 2 comments

我将 builtins.open = smart_open 后, 发现 smart_open 不支持 管道 pipe, 最小复现如下:

import os, sys
from megfile import smart_open as open

print ("The child will write text to a pipe and ")
print ("the parent will read the text written by child...")

# 文件描述符 r, w 用于读、写
r, w = os.pipe() 

processid = os.fork()
if processid:
    # 父进程
    # 关闭文件描述符 w
    os.close(w)
    r = open(r)
    print ("Parent reading")
    str = r.read()
    print ("text =", str)
    # sys.exit(0)
else:
    # 子进程
    os.close(r)
    w = open(w, 'w')
    print ("Child writing")
    w.write("Text written by child...")
    w.close()
    print ("Child closing")
    sys.exit(0)
Traceback (most recent call last):

  File "/home/dl/megvii/project/ai_asrs/jinyu_data_code/analysis_domain_gap.py", line 26, in <module>
    r = open(r)

  File "/home/dl/mygit/megfile/megfile/smart.py", line 436, in smart_open
    return SmartPath(path).open(mode, **options)

  File "/home/dl/mygit/megfile/megfile/smart_path.py", line 37, in __init__
    pathlike = self._create_pathlike(path)

  File "/home/dl/mygit/megfile/megfile/smart_path.py", line 64, in _create_pathlike
    protocol, path_without_protocol = cls._extract_protocol(path)

  File "/home/dl/mygit/megfile/megfile/smart_path.py", line 59, in _extract_protocol
    raise ProtocolNotFoundError('protocol not found: %r' % path)

ProtocolNotFoundError: protocol not found: 67

后续版本会支持

megfile 0.0.11 已发布,现已支持对 pipe 的读写