CloudShark Support

The upload API method

This method was introduced in CloudShark 1.2.485 and refers to version 1.0 of the API

Description

The upload API call provides a simple mechanism for remotely uploading files to the CloudShark appliance. The upload call returns the associated capture session which can then be used to link directly to the decode session. The capture session is returned in JSON format.

Optional parameters include filename, _tags, and comments. If filename is not specified, the file will be named after the browser’s User Agent and a unique timestamp. The additional_tags is a comma separated list of tags the file will be saved with. File comments may also be added at upload time by sending the comments parameter. See the curl examples below for a demonstration. NOTE that when using the PUT method, optional parameters must be appended to the URL rather than being specified as multipart/form-encoded data.

POST Usage

POST /api/v1/<token>/upload

Required parameters: the upload method requires either a url or file parameter. The form will either indicate “url=http://some-host/some-file.cap" or “file=<the actual contents of the file>” File contents must be encoded as multipart/form-data.

PUT Usage

Support for the PUT method was introduced in CloudShark 1.5.

PUT /api/v1/<token>/upload

Upload with PUT supports receiving a binary stream of data that is the contents of the file.

Expected Response

CloudShark returns the result of the upload call in JSON. It is an object representing the newly created capture session, or containing any errors that prevented the file from being imported.

{"filename":"traffic-2011-08-16-104829.cap","id":"f27d1494400c"}

Here is the full HTTP response with headers.

HTTP/1.1 200 OK
Server: nginx/0.9.4
Date: Tue, 16 Aug 2011 14:48:34 GMT
Content-Type: text/javascript; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive

40
{"filename":"traffic-2011-08-16-104829.cap","id":"f27d1494400c"}
0

The session id can be used to build a URL directly to the decode session.

https://www.cloudshark.org/captures/f27d1494400c

If the upload fails for any reason, the HTTP response may be either 4xx or 5xx. In this case the JSON body will contain the HTTP status along with an array of exceptions explaining the failure.

{"status":404,"exceptions":["API Token is missing or invalid"]}

Additional Parameters

Optional parameters can be set at the time of upload via the API:

  • additional_tags - a comma-separated list of tags to apply to the capture file
  • filename - specify the filename of the capture in CloudShark
  • comments - file-level comments

Examples

Below are some examples of calling the upload API in a few different languages. We are big fans of Ruby here at CloudShark, but we wanted to show how this can be implemented a variety of ways.

Curl

Curl can be used to upload local files and files available at specific URLs with a CloudShark API token. One of the nice things about curl is that it will do all the form encoding for you with the handy -F flag. A few simple examples are shown below.

Curl with a local file:
curl -F file=@filename.cap https://www.cloudshark.org/api/v1/<api-key>/upload
Curl with a URL:
curl -F url=http://path/to/capture/file https://www.cloudshark.org/api/v1/<api-key>/upload
Curl with a URL and HTTP authentication:
curl -F url=http://username:password@path/to/capture/file https://www.cloudshark.org/api/v1/<api-key>/upload
Include additional tags and change the filename:
curl -F file=@filename.cap -F additional_tags=tag1,tag2,tag3 -F filename=new_filename.cap https://www.cloudshark.org/api/v1/<api-key>/upload
Curl using PUT instead of POST:
curl --upload-file filename.cap https://www.cloudshark.org/api/v1/<api-key>/upload
Curl using PUT and adding comments:
curl --upload-file filename.cap https://www.cloudshark.org/api/v1/<api-key>/upload?comments=captured+from+NOC+23
Gotchas:

Be careful to URL-encode all your parameter values when using curl. For example anything that includes spaces should be sent with ‘+’ (plus-sign) instead. Also if specifying multiple URL parameters separated with the & sign, be sure to escape it ( &amp; ) so bash doesn’t interpret it as the end of the command.

Bash

While bash is not the best language to invoke a web API, it is possible to write simple scripts to upload files. Here is an example of using tcpdump to generate a capture file and then upload it to CloudShark.

#!/bin/sh
#
# bash script to use CloudShark Token API
#

UPLOAD=traffic-`date +%F-%H%M%S`.cap
CLOUDSHARKURL=https://cloudshark.myhost.example/api/v1/6d4ca6e11916dfc64c4d98c0c9fcea72/upload

# -- generate a capture file using tcpdump
echo "Generate a capture file"
tcpdump -i eth0 -c 100 -w $UPLOAD

# -- build a curl command
JSONID=`curl -F "file=@$UPLOAD" -F "additional_tags=script" $CLOUDSHARKURL | python -mjson.tool | grep id`

if [ "$JSONID" != "" ]; then

   # -- find the CloudShark ID for this session
   ID=`echo $JSONID | sed 's/:/ /1' | awk -F" " '{ print $2 }'| sed 's/\"//g'`

   echo "Uploaded $UPLOAD to CloudShark"

   # -- show a URL using the capture session in CloudShark
   echo "Please visit https://cloudshark.myhost.example/captures/$ID"
else
   echo "Could not upload capture to CloudShark"
fi

Ruby

For Ruby, Net::HTTP can be used to generate a POST. There is also a multipart form gem ‘multipart-post’ that makes it easy to POST. You will need to install the ‘mulipart-post’ gem first.

gem install multipart-post

Here is a simple example uploading an existing capture file.

#!/usr/bin/env ruby

require 'rubygems'
require 'json'
require 'net/http/post/multipart'

cloudshark = "cloudshark.xyz.example"
apikey = "6d4ca6e11916dfc64c4d98c0c9fcea72"

url = URI.parse('http://' + cloudshark + '/api/v1/' + apikey + '/upload/')

# -- upload traffic.cap
File.open("./traffic.cap") do |cap|
  req = Net::HTTP::Post::Multipart.new url.path,
    "file" => UploadIO.new(cap, "application/octet-stream", "traffic.cap")
  res = Net::HTTP.start(url.host, url.port) do |http|
    http.request(req)
  end

  puts "Received response #{res.body}"

  # -- parse the JSON response
  result = JSON.parse(res.body)

  # -- check for the session id
  if result.has_key? 'id'
    id = result["id"]
    puts "New capture uploaded with session id #{id}"
    puts "Access URL http://#{cloudshark}/captures/#{id}"
  else
    puts "Capture file was not uploaded"
  end

end

Java

Each of these Java clients will print out the CloudShark JSON response, or print out the Exception if a server error is encountered.

import java.lang.*;
import java.net.*;
import java.io.*;
class CloudSharkApiDemoUrl {
  public static void main(String[] args) {

    URL                 url;
    URLConnection   urlConn;
    DataOutputStream    printout;
    DataInputStream     input;

    try {
      url = new URL("https://www.cloudshark.org.myhost.example/api/v1/6d4ca6e11916dfc64c4d98c0c9fcea72/upload/");
      // Open the connection
      urlConn = url.openConnection();
      urlConn.setDoInput (true);
      urlConn.setDoOutput (true);
      urlConn.setUseCaches (false);
      // We are doing HTTP Form Encoding
      urlConn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
      // Open a DataOutputStream to write to the POST channel
      printout = new DataOutputStream (urlConn.getOutputStream ());
      // This is the CloudShark URL upload method, where we refer to another HTTP accessable PCAP resource
      // For this example, we are using a Wireshark.org Sample Capture for FTP over IPv6.
      String content = "url=" + URLEncoder.encode ("http://wiki.wireshark.org/SampleCaptures?action=AttachFile&amp;do=get&amp;target=FTPv6-2.cap");
      printout.writeBytes (content);
      printout.flush ();
      printout.close ();
      // Now we will read the response from CloudShark
      input = new DataInputStream (urlConn.getInputStream ());
      String str;
      while (null != ((str = input.readLine())))
      {
        System.out.println (str);
      }
      input.close ();
    }
    // This is a generic catch-all; production code should more-closely inspect errors.
    catch ( Exception e ){
      System.out.println (e);
    }
  }
}
Simple Java client upload by file method
import java.lang.*;
import java.net.*;
import java.io.*;
import com.myjavatools.web.*;
class CloudSharkApiDemoFile {
  public static void main(String[] args) {
    //ClientHttpRequest is available at http://www.devx.com/assets/sourcecode/7315.zip, Authored by Vlad Patryshev
    try {
      ClientHttpRequest chr=new ClientHttpRequest("https://www.cloudshark.org.myhost.example/api/v1/6d4ca6e11916dfc64c4d98c0c9fcea72/upload/");
      chr.setParameter("file",new File("packet.cap"));
      InputStream is = chr.post();
      Reader reader = new BufferedReader(  new InputStreamReader(is, "UTF-8") );
      char[] buffer = new char[1024];
      Writer writer = new StringWriter();
      int n;
      while ((n = reader.read(buffer)) != -1) {
        writer.write(buffer, 0, n);
      }
      is.close();
      System.out.println( writer.toString() );
    }
    // This is a generic catch-all; production code should more-closely inspect errors.
    catch ( Exception e ){
      System.out.println (e);
    }
  }
}

Windows PowerShell

Windows 8 users can natively capture packets using the netsh capture interface. With the Microsoft Message Analyzer tool, this proprietary format can be converted back into a pcap file and then uploaded using all native powershell cmdlets. With this one additional Microsoft download, a Windows 8 system is capable of pure packet capture and upload to CloudShark.

# CloudShark PowerShell Demo Script
# Free to distribute; Send us improvements and we'll credit and post!
# To use this script you must:
# (1) be in an Admin PowerShell
# (2) you must allow unsigned scripts: run
#     Set-ExecutionPolicy RemoteSigned
# (3) You must have Message Analyzer installed
# (Restart the Admin PowerShell after installing Message Analyzer
#     to load the New-PefTraceSession cmdlets)

# Set this to your CloudShark host
$cloudshark="edge.lan"
# Set this to your CloudShark upload API token
$tok="4fc5d18f2ae74f2c3b75f1879d343d00"

# Size of the capture file, in megabytes.
$maxsize=1

# Clean up any previous session data
If (Test-Path "C:\Users\$env:username\Desktop\OutFile.Cap"){
    Remove-Item "C:\Users\$env:username\Desktop\OutFile.Cap"
}

If (Test-Path "C:\Users\$env:username\Desktop\trace.etl"){
    Remove-Item "C:\Users\$env:username\Desktop\trace.etl"
}
netsh trace stop

# Start a new capture
netsh trace start capture=yes traceFile=C:\Users\$env:username\Desktop\trace.etl maxSize=$maxsize

$s = New-PefTraceSession -Path "C:\Users\$env:username\Desktop\OutFile.Cap" -SaveOnStop
$s | Add-PefMessageProvider -Provider "C:\Users\$env:username\AppData\Local\Temp\NetTraces\NetTrace.etl"
$s | Start-PefTraceSession

Invoke-RestMethod -uri http://$cloudshark/api/v1/$tok/upload -method PUT -inFile C:\Users\$env:username\Desktop\OutFile.Cap

About CloudShark

CloudShark is made by QA Cafe, a technology company based in Portsmouth, NH. Our passion for packet captures has grown out of our other product CDRouter.

Get in touch via our Contact us page or by following us on your favorite service: