kubernetes-client/java

The copyFileToPodAsync method causes the process to fail to close!

swzaaaaaaa opened this issue · 5 comments

Describe the bug
Hello. I refer to the link "https://github.com/kubernetes-client/java/blob/master/util/src/main/java/io/kubernetes/client/Copy.java" to write file upload Code, found that the code will get stuck in "proc.waitFor(); "This step. I think it is not the output stream that can not be closed all the time. Is there any good method?
I tried to comment out "proc.waitFor()" and "proc.destroy()" in full, and I was able to complete the file upload, but this should cause problems such as memory leaks.

Kubernetes Version
e.g. 1.15.0

Java Version
e.g. Java 8

To Reproduce


public void copyFileToPod(
      String namespace, String pod, String container, byte[] src, Path destPath)
      throws ApiException, IOException {
    try {
      int exit = copyFileToPodAsync(namespace, pod, container, src, destPath).get();
      if (exit != 0) {
        throw new IOException("Copy failed: " + exit);
      }
    } catch (InterruptedException | ExecutionException ex) {
      throw new IOException(ex);
    }
  }

  public Future<Integer> copyFileToPodAsync(
      String namespace, String pod, String container, byte[] src, Path destPath)
      throws ApiException, IOException {
    // Run decoding and extracting processes
    final Process proc = execCopyToPod(namespace, pod, container, destPath);

    try (ArchiveOutputStream archiveOutputStream =
        new TarArchiveOutputStream(proc.getOutputStream())) {

      ArchiveEntry tarEntry = new TarArchiveEntry(new File(destPath.getFileName().toString()));
      ((TarArchiveEntry) tarEntry).setSize(src.length);

      archiveOutputStream.putArchiveEntry(tarEntry);
      Streams.copy(new ByteArrayInputStream(src), archiveOutputStream);
      archiveOutputStream.closeArchiveEntry();

      return new ProcessFuture(proc);
    }
  }

  private Process execCopyToPod(String namespace, String pod, String container, Path destPath)
      throws ApiException, IOException {
    // TODO: This assumes Linux and won't work on Windows Containers (for many reasons...)
    String parentPath = destPath.getParent() != null ? destPath.getParent().toString() : ".";
    parentPath = parentPath.replace("\\", "/");
    return this.exec(
        namespace,
        pod,
        new String[] {"sh", "-c", "tar -xmf - -C " + parentPath},
        container,
        true,
        false);
  }

  private static class ProcessFuture implements Future<Integer> {
    private Process proc;

    ProcessFuture(Process proc) {
      this.proc = proc;
    }

    // TODO: support cancelling?
    public boolean cancel(boolean interupt) {
      return false;
    }

    public boolean isCancelled() {
      return false;
    }

    public Integer get() throws InterruptedException {
      **proc.waitFor(); /////here**
      proc.destroy();
      return proc.exitValue();
    }

    public Integer get(long timeout, TimeUnit unit) throws InterruptedException {
      proc.waitFor(timeout, unit);
      proc.destroy();
      return proc.exitValue();
    }

    public boolean isDone() {
      return proc.isAlive();
    }
  }

Expected behavior
A clear and concise description of what you expected to happen.

KubeConfig
If applicable, add a KubeConfig file with secrets redacted.

You can manually close it after the upload is completed

Future<Integer> future = copy.copyFileToPodAsync(namespace, podInfo.getPodName(), null, Paths.get(srcPath), Paths.get(descPath));
  future.get(1, TimeUnit.NANOSECONDS); 

You can manually close it after the upload is completed

Future<Integer> future = copy.copyFileToPodAsync(namespace, podInfo.getPodName(), null, Paths.get(srcPath), Paths.get(descPath));
  future.get(1, TimeUnit.NANOSECONDS); 

Hello, I use this method, but it will report an Error every time: "Error on flush, java.io.IOException: Socket is closed!" Is this normal? Where the file is finished uploading.

You can manually close it after the upload is completed

Future<Integer> future = copy.copyFileToPodAsync(namespace, podInfo.getPodName(), null, Paths.get(srcPath), Paths.get(descPath));
  future.get(1, TimeUnit.NANOSECONDS); 

Hello, I use this method, but it will report an Error every time: "Error on flush, java.io.IOException: Socket is closed!" Is this normal? Where the file is finished uploading.

Streams.copy(new ByteArrayInputStream(src), archiveOutputStream);
This line of code executes the upload,
Do you have a complete stack, or try uploading a small file

The same is true when I uploaded a small file (less than 1M). I suspect that pod could not know the end of the uploaded file, so it would hang on forever, which would cause this problem when we proactively closed.
As the question says “https://github.com/kubernetes-client/java/issues/1822”

future.get(1, TimeUnit.NANOSECONDS); can solve resolution.