eldur/jwbf

AllPageTitles constructor's "from" attribute is ineffective

Closed this issue · 9 comments

minhlab commented

prepareCollection doesn't happen to use "from" when it is first called.

  protected HttpAction prepareCollection() {
    return generateRequest(nextPageInfoOpt(), prefix, rf, 
        MWAction.createNsString(namespaces));

  }
eldur commented

can you give me more details when it is called?

If I look at AllPageTitlesIntegTest.java, there are 3 mocked requests. Each of them can only be requested once. The inital request has no "from" value, but all requests are called.

minhlab commented

I mean... Just write "for (String title : new AllPageTitles(bot, from)) {
System.out.println(title); }" and you will see that the from argument is
disregarded.

The problem is precisely at the first request to server which doesn't use
the "from" field.

On Mon, Jul 14, 2014 at 10:47 PM, Loki notifications@github.com wrote:

can you give me more details when it is called?

If I look at AllPageTitlesIntegTest.java
https://github.com/eldur/jwbf/blob/master/src/integration-test/java/net/sourceforge/jwbf/mediawiki/actions/queries/AllPageTitlesIntegTest.java#L21,
there are 3 mocked requests. Each of them can only be requested once. The
inital request has no "from" value, but all requests are called.

—
Reply to this email directly or view it on GitHub
#19 (comment).

eldur commented

Hmm if a from value is given it will be send to server ... you don't have to set it..

for (String title : new AllPageTitles(bot, null)) {
  System.out.println(title);
}

or shorter

for (String title : new AllPageTitles(bot)) {
  System.out.println(title);
}

the from constructor param is for later (manually) continuation.

minhlab commented

Yes, I would love to continue where my program stopped. I was dealing with
hundreds thousand articles and all too often I realized that I needed to
modify a line in my code or my program ran into a bug. Each time I needed
to wait for it to traverse the list of all titles from the start.

On Mon, Jul 14, 2014 at 11:16 PM, Loki notifications@github.com wrote:

Hmm if a from value is given it will be sent to server ... you don't have
to set it..

for (String title : new AllPageTitles(bot, null)) {
System.out.println(title);}

or shorter

for (String title : new AllPageTitles(bot)) {
System.out.println(title);}

the from constructor param is for later (manually) continuation.

—
Reply to this email directly or view it on GitHub
#19 (comment).

eldur commented

Can you give me a brief description what the main goals of your program are? It sounds there are other ways ... btw: are you working on live wikipedia?

eldur commented

If you don't need to query live data, there are also tools to work with full wikipedia dump files like JWPL (I've never tried it)

minhlab commented

Yes, I was working on live Vietnamese Wikipedia. My program search articles
for a particular type of problems and suggest a fix that I need to proof
read. I'll think of a solution though. Thank you!

On Mon, Jul 14, 2014 at 11:46 PM, Loki notifications@github.com wrote:

Can you give me a brief description what the main goals of your program
are? It sounds there are other ways ... btw: are you working on live
wikipedia?

—
Reply to this email directly or view it on GitHub
#19 (comment).

eldur commented

I tend to close this task (because AllPageTitles isn't ineffective), okay?