guest20/HTTP-Request-FromCurl

create a HTTP::Request from a curl command line

NAME

HTTP::Request::FromCurl - create a HTTP::Request from a curl command line

SYNOPSIS

my $req = HTTP::Request::FromCurl->new(
    # Note - curl itself may not appear
    argv => ['https://example.com'],
);

my $req = HTTP::Request::FromCurl->new(
    command => 'https://example.com',
);

my $req = HTTP::Request::FromCurl->new(
    command_curl => 'curl -A mycurl/1.0 https://example.com',
);

my @requests = HTTP::Request::FromCurl->new(
    command_curl => 'curl -A mycurl/1.0 https://example.com https://www.example.com',
);
# Send the requests
for my $r (@requests) {
    $ua->request( $r->as_request )
}

RATIONALE

curl command lines are found everywhere in documentation. The Firefox developer tools can also copy network requests as curl command lines from the network panel. This module enables converting these to Perl code.

METHODS

->new

my $req = HTTP::Request::FromCurl->new(
    # Note - curl itself may not appear
    argv => ['--user-agent', 'myscript/1.0', 'https://example.com'],
);

my $req = HTTP::Request::FromCurl->new(
    # Note - curl itself may not appear
    command => '--user-agent myscript/1.0 https://example.com',
);

The constructor returns one or more HTTP::Request::CurlParameters objects that encapsulate the parameters. If the command generates multiple requests, they will be returned in list context. In scalar context, only the first request will be returned. Note that the order of URLs between --url and unadorned URLs will be changed in the sense that all unadorned URLs will be handled first.

my $req = HTTP::Request::FromCurl->new(
    command => '--data-binary @/etc/passwd https://example.com',
    read_files => 1,
);

Options

• argv

  An arrayref of commands as could be given in @ARGV.

• command

  A scalar in a command line, excluding the curl command

• command_curl

  A scalar in a command line, including the curl command

• read_files

  Do read in the content of files specified with (for example) --data=@/etc/passwd. The default is to not read the contents of files specified this way.

GLOBAL VARIABLES

%default_headers

Contains the default headers added to every request

@option_spec

Contains the Getopt::Long specification of the recognized command line parameters.

The following curl options are recognized but largely ignored:

• --dump-header

• --include

• --location

• --progress-bar

• --show-error

• --fail

• --silent

• --verbose

• --junk-session-cookies

  If you want to keep session cookies between subsequent requests, you need to provide a cookie jar in your user agent.

• --next

  Resetting the UA between requests is something you need to handle yourself

• --parallel

• --parallel-immediate

• --parallel-max

  Parallel requests is something you need to handle in the UA

METHODS

->squash_uri( $uri )

my $uri = HTTP::Request::FromCurl->squash_uri(
    URI->new( 'https://example.com/foo/bar/..' )
);
# https://example.com/foo/

Helper method to clean up relative path elements from the URI the same way that curl does.

LIVE DEMO

https://corion.net/curl2lwp.psgi

KNOWN DIFFERENCES

Incompatible cookie jar formats

Until somebody writes a robust Netscape cookie file parser and proper loading and storage for HTTP::CookieJar, this module will not be able to load and save files in the format that Curl uses.

Loading/saving cookie jars is the job of the UA

You're expected to instruct your UA to load/save cookie jars:

use Path::Tiny;
use HTTP::CookieJar::LWP;

if( my $cookies = $r->cookie_jar ) {
    $ua->cookie_jar( HTTP::CookieJar::LWP->new()->load_cookies(
        path($cookies)->lines
    ));
};

Different Content-Length for POST requests

Different delimiter for form data

The delimiter is built by HTTP::Message, and curl uses a different mechanism to come up with a unique data delimiter. This results in differences in the raw body content and the Content-Length header.

MISSING FUNCTIONALITY

• File uploads / content from files

  While file uploads and reading POST data from files are supported, the content is slurped into memory completely. This can be problematic for large files and little available memory.

• Mixed data instances

  Multiple mixed instances of --data, --data-ascii, --data-raw, --data-binary or --data-raw are sorted by type first instead of getting concatenated in the order they appear on the command line. If the order is important to you, use one type only.

• Multiple sets of parameters from the command line

  Curl supports the --next command line switch which resets parameters for the next URL.

  This is not (yet) supported.

SEE ALSO

LWP::Curl

LWP::Protocol::Net::Curl

LWP::CurlLog

HTTP::Request::AsCurl - for the inverse function

The module HTTP::Request::AsCurl likely also implements a much better version of ->as_curl than this module.

https://github.com/NickCarneiro/curlconverter - a converter for multiple target languages

REPOSITORY

The public repository of this module is http://github.com/Corion/HTTP-Request-FromCurl.

SUPPORT

The public support forum of this module is https://perlmonks.org/.

BUG TRACKER

Please report bugs in this module via the Github bug queue at https://github.com/Corion/HTTP-Request-FromCurl/issues

AUTHOR

Max Maischein corion@cpan.org

COPYRIGHT (c)

Copyright 2018-2022 by Max Maischein corion@cpan.org.

LICENSE

This module is released under the same terms as Perl itself.