wspl/creeper

Feature: Next Page Node - functional node for directing next page

wspl opened this issue · 0 comments

wspl commented
page = "http://example.com/info?page=1"
demo[]: page -> $(".example")
    text: $(".title").html
    @next: $("a.next").href

I am thinking about another method for page number directing, that is simulating the operation of the user click on the next page. We can add a @next node to indicates the next page link. Page director would switch to next page automatically when current page has no more content.

New grammar features - Functional node: For assisting crawling. Node name start with @. It is less readable than private nodes.