Today, when I re-used the package "GoogleNews" in Python that can automatically return search results (you can refer to [Python] Using "GoogleNews" package to get the Google News that I wrote before), it may be because I upgraded this package The version of, there is no problem in the English search, but the search result in the Chinese search is not returned, and there is no error.
After I traced the source code, I found that the developer made a brand-new adjustment. Maybe it was also in response to the Google News update.
After I tested it, I finally succeeded in getting the Chinese search to return results. But this time things started to make me think, maybe after all, I still have to write a crawler myself more thoroughly.
Below, I will briefly record how the current version should be searched in Chinese. (GoogleNews 1.3.7)
Google News Chinese search is invalid
First, I used the English search test.
from GoogleNews import GoogleNews googlenews = GoogleNews() googlenews.search('US Stock') result = googlenews.gettext() print(len(result)) for n in range(len(result)): print(n) print(result[n])
Output:
10
0
US stocks end higher, snap two-day losing streak on crude …
1
Bitcoin Price Breaks $7000 as US Stock Market Sees Minor …
2
US stocks close higher as oil prices face pressure
3
Why a 'return to normal' could mean disaster for the stock market
4
US stock futures bounce after two days of oil-led rout
5
Airlines and Automakers Consider How to Reopen: Live …
6
Stock snap two-day losing streak
7
Oil prices rebound from 21-year low; gasoline futures rally on …
8
Gung-Ho Fed Exposes 'Gigantic Hole' That May Rattle Stock …
9
Stock market live Wednesday: Dow up 450, oil rebounds from …It seems that there is no problem with searching in English at the moment… Well, after all, it is the main function developed by the author! It shouldn't even fail this function.
Then I tested Chinese search.
from GoogleNews import GoogleNews googlenews = GoogleNews() googlenews.search('美股') result = googlenews.gettext() print(len(result)) for n in range(len(result)): print(n) print(result[n])
Output:
0No results were returned this time. The magic is that this time is different from before, the previous error disappeared. We need to adjust the source code of the GoogleNews package again.
The location of the source code is in __init__.py under site-packages/GoogleNews/ of your installation package.
I researched it. This time the crawler package updated the language distinction, which means that we need another command to search for the language; in addition, the previous bug that could not search for Chinese still exists, but the line code has been put into error handling "try-except", so no error will be reported.
Similarly, import at the top of __init__.py:
import string
Then find this line code:
self.url = "https://www.google.com/search?q={}&lr=lang_{}&tbs=lr:lang_1{},qdr:{}&tbm=nws&start={}".format(self.__key,self.__lang,self.__lang,self.__period,(10 * (page - 1)))
Add a line of code below to become:
self.url = "https://www.google.com/search?q={}&lr=lang_{}&tbs=lr:lang_1{},qdr:{}&tbm=nws&start={}".format(self.__key,self.__lang,self.__lang,self.__period,(10 * (page - 1))) self.url = urllib.request.quote(self.url, safe=string.printable)
Then go back to our code, set the language to "cn", and search in Chinese:
from GoogleNews import GoogleNews googlenews = GoogleNews('cn') googlenews.search('美股') result = googlenews.gettext() print(len(result)) for n in range(len(result)): print(n) print(result[n])
Output:
10
0
美股道瓊大漲456點重啟經濟的樂觀預期升溫
1
〈美股盤後〉油價反彈！更多刺激方案出台德儀領費半猛漲近6%
2
坐穩了！小摩：明年上半年美股將再創歷史新高
3
這隻黑天鵝來亂！美股挫台股也殺不下去
4
【不斷更新】油價反彈美股上攻道瓊漲逾470點
5
美銀：從VIX歷史規律來看S&P新一輪下跌正在路上
6
布局美股搶賺轉機財
7
《美股掃瞄》美股消息面：美伊緊張＋產油減少美國期油飆漲19.1％
8
《美股》營收、用戶增長意外強勁Snap盤後飆升逾20％
9
〈台股盤前〉美股費半強彈回穩短線不悲觀電子股有望扮演反攻要角It works fine on my environment.