jarun/googler

all searches give No results

kurokirasama opened this issue · 27 comments

hello,
i haven't used googler in a while but when i tried again today all my searches gives No results.

Here is the file:
googler -d matlab

I'm in ubuntu 18.04, Python 2.7.15rc1, terminator 1.91 and GNOME Shell 3.28.3

PD: I tried the master.zip file you suggest, but now is empty.

Can confirm. I do get one result for "matlab":

$ googler --np matlab
1 MATLAB (@MATLAB) · Twitter
https://twitter.com/MATLAB?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor

But most simple queries turn up no results. Apparently they broke our parser.

I'll take a look tonight.

Which is irrelevant. Please keep the discussion on topic.

jarun commented

--noua

~/G/googler$ ./googler --noua -d hello
[DEBUG] googler version 3.7
[DEBUG] Python version 3.6.6
[DEBUG] User Agent is disabled
[DEBUG] Connecting to new host www.google.com
[DEBUG] Fetching URL /search?ie=UTF-8&oe=UTF-8&q=hello
[DEBUG] Cookie: 1P_JAR=2018-10-05-21
[DEBUG] Response body written to '/tmp/googler-response-vc2oiqip'.

1 hello - YouTube  [www.youtube.com]
22 Aug 2017 - 3 min - Uploaded by Lauv about making a big move, falling in love for the first time, and figuring out your own identity while ...

2 Hello (Adele song) - Wikipedia  [en.wikipedia.org]
"Hello" is a song by English singer Adele, released on 23 October 2015 by XL Recordings as the lead single from her third studio album, 25 (2015). Adele ...

3 hello network  [hello.com]
Connect with people who share your interests. Discover communities you love. Join fun conversations.

4 hello - Wiktionary  [en.wiktionary.org]
The greeting hello is among the most generic and neutral in use. It may be heard in nearly all social situations and in nearly all walks of life, and is unlikely to ...

5 Hello Design  [www.hellodesign.com]
Hello is a creative agency driven to craft worthy experiences. We believe everything will be digital—surrounding us like the air we breathe. Hello is ...

6 hello | Definition of hello in English by Oxford Dictionaries  [en.oxforddictionaries.com]
Definition of hello - used as a greeting or to begin a telephone conversation.

7 hello Meaning in the Cambridge English Dictionary  [dictionary.cambridge.org]
hello definition: 1. used when meeting or greeting someone: 2. something that is said at the beginning of a phone conversation: 3. something that is said to ...

8 Hello Games  [www.hellogames.org]
We are Hello Games. We make games! Hello! Follow us on TWITTER, read our BLOG! Like us on FACEBOOK! Do you want to post YouTube videos of our ...

9 HELLO! - Daily royal, celebrity, fashion, beauty & lifestyle news  [www.hellomagazine.com]
HELLO! brings you the latest celebrity & royal news from the UK & around the world, magazine exclusives, fashion, beauty, lifestyle news, celeb babies, ...

googler (? for help) 
googler (? for help) 
~/G/googler$ ./googler -d hello
[DEBUG] googler version 3.7
[DEBUG] Python version 3.6.6
[DEBUG] Connecting to new host www.google.com
[DEBUG] Fetching URL /search?ie=UTF-8&oe=UTF-8&q=hello
[DEBUG] Cookie: 1P_JAR=2018-10-05-21
[DEBUG] Response body written to '/tmp/googler-response-qfztq74o'.
No results.
googler (? for help) 
googler (? for help) 
~/G/googler$
jarun commented

Somehow the User Agent is not liked ;).

--noua uses a legacy layout they didn't bother to break. Maybe we could just use that as default. Do you remember what's missing with --noua?

jarun commented

We just don't send the UA. That's it.

I mean, does it omit certain results? There are certainly no smart cards, but I think we're skipping the cards anyway.

jarun commented

No it doesn't.

Easy fix then.

jarun commented

Can you try to change some of the other header fields?

jarun commented

I am thinking if the Upgrade-Insecure-Requests has any effect.

I can't because I'm on my phone now ;)

jarun commented

I am trying it out now.

jarun commented

Try e704598.

jarun commented

@zmwangx BTW, do take a look at the data they are returning with FF UA. If it's a parser issue we should fix that.

jarun commented

Seems like the format has changed for the url and title...

I'm facing the same problem. It's not returning any results for me.

$ googler hello
No results.

--noua has been deprecated, so can't go through that route either.

$ googler --noua hello
[WARNING] --noua option has been deprecated and has no effect (see #284)
No results.

I installed googler and it's not working I tried to debug,
Debug result-
Screenshot from 2019-11-14 07-19-01
Where ddgr works pretty fine
Result-
Screenshot from 2019-11-14 07-22-04

jarun commented

Install v3.9.

@Avileox I decided to just use ddgr in my scripts rather than googler until the issue gets fixed. In any case, thank you for the programs, @jarun. They've really made my life easier.

@jarun You have to update the apt-get policy of the following package
It downloads an older version of the repository in kali linux
Screenshot from 2019-11-14 08-22-56
I will download v3.9

googler -d hello
[DEBUG] googler version 3.9
[DEBUG] Python version 3.7.5
[DEBUG] Connecting to new host www.google.com
[DEBUG] Fetching URL /search?ie=UTF-8&oe=UTF-8&q=hello&sei=6L1uaA11EeqMX9CBetSyqA
[DEBUG] Cookie: 1P_JAR=2019-11-22-22
[DEBUG] Response body written to '/var/folders/45/3lbn2njj4q13dmn7b5bnwc280000gn/T/googler-response-b26dfh61.html'.
No results.

There are valid results in the "/var/folders/45/3lbn2njj4q13dmn7b5bnwc280000gn/T/googler-response-b26dfh61.html" file but googler is stating that there are "No results"

In looking at your python file I notice that you try to import "readline" on line 48. Yesterday, when I was trying to use hunspell it could not find libreadline.so so I re-installed (e.g. brew reinstall hunspell), which likely pulled in a new libreadline.so because that program started working after the reinstall. Maybe something is going on with this library?

Not sure why this is closed, I'm running into the same issue on fedora 30.

> googler -n 5 --noprompt --noua -d hello
[DEBUG] googler version 3.9
[DEBUG] Python version 3.7.5
[WARNING] --noua option has been deprecated and has no effect (see #284)
[DEBUG] Connecting to new host www.google.com
[DEBUG] Fetching URL /search?ie=UTF-8&num=5&oe=UTF-8&q=hello&sei=fKyT5hoaEeqHvrQumaPgPA
[DEBUG] Cookie: 1P_JAR=2019-12-09-00
[DEBUG] Response body written to '/tmp/googler-response-yh8zbp2g.html'.
No results.

Response seems valid...

> w3m -dump /tmp/googler-response-yh8zbp2g.html
Google
Google
[hello               ]
×
Refresh (0 sec) /search?q=hello&num=5&ie=UTF-8&oe=UTF-8&gbv=1&sei=
nZTtXeTZF9Se9QOZ-6-4Cw
Please click here if you are not redirected within a few seconds.
AllImagesVideosMaps News Shopping Books
Search tools
Any time

  • Any time
  • Past hour
  • Past 24 hours
  • Past week
  • Past month
  • Past year

All results

  • All results
  • Verbatim

 
[yH5BAEKAAE]
6:07
Adele - Hello - YouTube
https://www.youtube.com/watch?v=YQHsXMglC9A
See results about
 
Hello
Hello is a salutation or greeting in the English language. It is first attested
in writing from 1826.
[yH5BAEKAAE]
 
Hello - YouTube
https://www.youtube.com › watch
 
Video for hello
Jun 23, 2016 · Hello, it's me I was wondering if after all these years you'd
like to meet To go over everything ...Duration: 4:56 Posted: Jun 23, 2016
 
Hello (Adele song) - Wikipedia
https://en.wikipedia.org › wiki › Hello_(Adele_song)
"Hello" is a song by British singer-songwriter Adele, released on 23 October
2015 by XL Recordings as the lead single from her third studio album, 25
(2015). Writing and composition · Release and reception · Chart performance · 
Music video Released: 23 October 2015 Genre: Soul Producer(s): Greg Kurstin
Songwriter(s): Adele Adkins; Greg Kurstin
 
hello - Wiktionary
https://en.wiktionary.org › wiki › hello
Hello (first attested in 1833), from holla, hollo (attested 1588). This variant
of hallo is often credited to Thomas Edison as a coinage for telephone use, but
its ...
 
Hello | Definition of Hello at Dictionary.com
https://www.dictionary.com › browse › hello
(used derisively to question the comprehension, intelligence, or common sense
of the person being addressed): You're gonna go out with him? Hello!
Related searches

And current python3 modules:

> pip3 list
Package            Version   
------------------ ----------
argcomplete        1.9.5     
asn1crypto         0.24.0    
Beaker             1.10.0    
blivet             3.1.4     
blivet-gui         2.1.11    
Brlapi             0.6.7     
cffi               1.11.5    
chardet            3.0.4     
chrome-gnome-shell 0.0.0     
coverage           4.5.1     
cryptography       2.6.1     
cupshelpers        1.0       
dbus-python        1.2.8     
decorator          4.3.0     
distro             1.4.0     
fros               1.1       
gpg                1.12.0    
humanize           0.5.1     
idna               2.7       
langtable          0.0.43    
libvirt-python     5.1.0     
Mako               1.1.0.dev0
MarkupSafe         1.1.1     
ntplib             0.3.3     
olefile            0.46      
ordered-set        2.0.2     
Paste              2.0.3     
pid                2.2.3     
Pillow             5.4.1     
pip                19.0.3    
ply                3.11      
productmd          1.23      
pwquality          1.4.0     
pyalsa             1.1.6     
pycairo            1.18.1    
pycparser          2.14      
pycrypto           2.6.1     
pycups             1.9.74    
pycurl             7.43.0.2  
pydbus             0.6.0     
pyenchant          2.0.0     
PyGObject          3.32.2    
pyinotify          0.9.6     
pykickstart        3.20      
pyOpenSSL          19.0.0    
pyparted           3.11.2    
PySocks            1.6.8     
python-augeas      0.5.0     
python-dateutil    2.8.0     
python-meh         0.47      
pytz               2018.5    
pyudev             0.21.0    
pyxdg              0.26      
requests           2.22.0    
requests-file      1.4.3     
requests-ftp       0.3.1     
rpm                4.14.2.1  
selinux            2.9       
sepolicy           1.1       
setools            4.1.1     
setuptools         40.8.0    
simpleline         1.4       
six                1.12.0    
slip               0.6.4     
slip.dbus          0.6.4     
sos                3.8       
systemd-python     234       
Tempita            0.5.1     
urllib3            1.25.7    

Any suggestions?
Thanks!

jarun commented

@paxri01 when you take the initiative to come to a project page and update a defect, also consider checking out the latest release and confirming if that works.

jarun commented

I am locking these old conversations as these distract people and they miss out the most important part: trying the latest release.