Record all outgoing/incoming transmissions

[dgmyrek]

Here's what shows up in the logs when I try this request:

~/Git/web-api-commander/out $ java -Dlog4j.configuration=file:/home/dylan/Git/web-api-commander/out/log4j.xml -jar web-api-commander.jar --readEntities --inputFile metadata_file --outputFile outputfile2 --limit -1 --uri "https://sparkapi.com/Reso/OData/Property?\$count=true&\$select=ListingKey,ListingId,PoolFeatures,PropertyType" --bearerToken [redacted] --useEdmEnabledClient --serviceRoot "https://sparkapi.com/Reso/OData"

[INFO] [11:49:03.015] Reading all entities from: https://sparkapi.com/Reso/OData/Property?$count=true&$select=ListingKey,ListingId,PoolFeatures,PropertyType [ERROR] [11:49:05.488] ERROR: readEntities could not continue. org.apache.olingo.client.api.http.HttpClientException: java.net.UnknownHostException: standards.reso.org: Name or service not known

Not really enough information to determine what's going on here, but at first glance one would assume there's a misconfiguration or server issue related to sparkapi.com. Would be great if we could include additional details in the logging - specifically the request to standards.reso.org that is actually failing here. At the very least any HTTP request made by commander should be included in the log.

Running without logging returns this error at the end of the console dump:

HTTP header being sent: Accept: application/xml HTTP header being sent: Content-Type: application/xml HTTP header being sent: OData-MaxVersion: 4.0 HTTP header being sent: OData-Version: 4.0 Get connection for route {}->http://standards.reso.org Connection org.apache.http.impl.conn.DefaultClientConnection@4efc180e closed Connection org.apache.http.impl.conn.DefaultClientConnection@4efc180e shut down Releasing connection org.apache.http.impl.conn.ManagedClientConnectionImpl@bd4dc25 Connection 0.0.0.0:42301<->76.10.121.154:443 closed ERROR: readEntities could not continue. org.apache.olingo.client.api.http.HttpClientException: java.net.UnknownHostException: standards.reso.org: Name or service not known

[darnjo]

In the current release of the Commander I've changed the default logging level to INFO rather than ALL, as was previously configured, in order to avoid duplicating the data transfer in the log.

The option to use more verbose or different logging options can be passed as an argument to the application, and you can provide a custom log4j.properties file for logging.

For instance, if you wanted to log to a file rather than piping the output of what's on the console to a file, this could be specified by passing a command line arg to a different logging file, which you can define locally and use to debug. There's more information about log4j.properties file here.

In this case, you'd want a log4j.properties file (name is arbitrary) similar to the following:

log4j.rootLogger=ALL, Console
log4j.appender.Console=org.apache.log4j.ConsoleAppender
log4j.appender.Console.layout=org.apache.log4j.PatternLayout
log4j.appender.Console.layout.conversionPattern=%m%n

which you can then pass to the Commander as follows:

 java -Dlog4j.configuration=file:log4j.properties -jar <... remaining arguments>

It's important to note that -Dlog4j.configuration=file:log4j.properties must contain a path to the file. In the case above, the file was in the same directory as the Java executable, but you'll need to change that if you're using a different directory.