/chromedp

Package chromedp is a faster, simpler way to drive browsers (Chrome, Edge, Safari, Android, etc) without external dependencies (ie, Selenium, PhantomJS, etc) using the Chrome Debugging Protocol.

Primary LanguageGoMIT LicenseMIT

About chromedp Build Status Coverage Status

Package chromedp is a faster, simpler way to drive browsers in Go using the Chrome Debugging Protocol (for Chrome, Edge, Safari, etc) without external dependencies (ie, Selenium, PhantomJS, etc).

NOTE: chromedp's API is currently unstable, and may change at a moments notice. There are likely extremely bad bugs lurking in this code. CAVEAT USER.

Installation

Install in the usual way:

go get -u github.com/knq/chromedp

Usage

Below is a simple Google search performed using chromedp (taken from examples/simple):

This example shows logic for a simple search for a known website, clicking on the right link, and then taking a screenshot of a specific element on the loaded page and saving that to a local file on disk.

// examples/simple/main.go
package main

import (
	"context"
	"fmt"
	"io/ioutil"
	"log"
	"time"

	cdp "github.com/knq/chromedp"
	cdptypes "github.com/knq/chromedp/cdp"
)

func main() {
	var err error

	// create context
	ctxt, cancel := context.WithCancel(context.Background())
	defer cancel()

	// create chrome instance
	c, err := cdp.New(ctxt, cdp.WithLog(log.Printf))
	if err != nil {
		log.Fatal(err)
	}

	// run task list
	var site, res string
	err = c.Run(ctxt, googleSearch("site:brank.as", "Home", &site, &res))
	if err != nil {
		log.Fatal(err)
	}

	// shutdown chrome
	err = c.Shutdown(ctxt)
	if err != nil {
		log.Fatal(err)
	}

	// wait for chrome to finish
	err = c.Wait()
	if err != nil {
		log.Fatal(err)
	}

	log.Printf("saved screenshot from search result listing `%s` (%s)", res, site)
}

func googleSearch(q, text string, site, res *string) cdp.Tasks {
	var buf []byte
	sel := fmt.Sprintf(`//a[text()[contains(., '%s')]]`, text)
	return cdp.Tasks{
		cdp.Navigate(`https://www.google.com`),
		cdp.WaitVisible(`#hplogo`, cdp.ByID),
		cdp.SendKeys(`#lst-ib`, q+"\n", cdp.ByID),
		cdp.WaitVisible(`#res`, cdp.ByID),
		cdp.Text(sel, res),
		cdp.Click(sel),
		cdp.WaitVisible(`a[href="/brankas-for-business"]`, cdp.ByQuery),
		cdp.WaitNotVisible(`.preloader-content`, cdp.ByQuery),
		cdp.Location(site),
		cdp.ScrollIntoView(`.banner-section.third-section`, cdp.ByQuery),
		cdp.Sleep(2 * time.Second), // wait for animation to finish
		cdp.Screenshot(`.banner-section.third-section`, &buf, cdp.ByQuery),
		cdp.ActionFunc(func(context.Context, cdptypes.Handler) error {
			return ioutil.WriteFile("screenshot.png", buf, 0644)
		}),
	}
}

Please see the examples directory for some more examples, and please refer to the GoDoc API listing for a summary of the API and Actions.

Links + Resources

TODO

  • Move timeouts to context (defaults)
  • Implement more query selector options (allow over riding context timeouts)
  • Contextual actions for "dry run" (or via an accumulator?)
  • Network loader / manager
  • Profiler