Fill form using python
I have searched the web for this topic. Here is the collection of some nice links and questions.
Nothing is mine.
import cookielib
import socket
import urllib
import urllib2
import re,os
import sys #read more about this module
if not os.path.exists("Girls"):
os.mkdir("Girls")
#source = urllib.urlopen("http://thechive.com/2012/06/12/beautiful-girls-40-photos/").read()
f = open("tmp.html","r")
source = f.read()
imgs = re.findall("img.*src=\"(.*)\?",source)
print imgs
for img in imgs:
print img
filename = "Girls/"+img.split("/")[-1].split("?")[0]+".jpg"
print filename
if not os.path.exists(filename):
urllib.urlretrieve(img,filename)
1.http://www.blog.pythonlibrary.org/2012/06/08/python-101-how-to-submit-a-web-form/
Here the author shows the use of urllib, urllib2, webbrowser and requests to do the job.
When you use urllib2
import urllib2 as urlget
The desired output of the code will be that of the url http://stackoverflow.com/search?q=hellojai
>>>url = "http://stackoverflow.com/search"
>>> data = {'q':"hellojai"}
>>> data = urllib.urlencode(data)
>>> data = urlget.urlopen(url,data)
see for the status code.
data.code
data.getcode()
data.message
FOR headers
1.data.headers.headers
2.data.url
For the Requests it wil like this
query = {'q':"hellojai"}
>>> datadata = requests.get(url,params=query)
Note : params is required.
This is how we make get request.
For post in both
Nothing is mine.
import cookielib
import socket
import urllib
import urllib2
import re,os
import sys #read more about this module
if not os.path.exists("Girls"):
os.mkdir("Girls")
#source = urllib.urlopen("http://thechive.com/2012/06/12/beautiful-girls-40-photos/").read()
f = open("tmp.html","r")
source = f.read()
imgs = re.findall("img.*src=\"(.*)\?",source)
print imgs
for img in imgs:
print img
filename = "Girls/"+img.split("/")[-1].split("?")[0]+".jpg"
print filename
if not os.path.exists(filename):
urllib.urlretrieve(img,filename)
1.http://www.blog.pythonlibrary.org/2012/06/08/python-101-how-to-submit-a-web-form/
Here the author shows the use of urllib, urllib2, webbrowser and requests to do the job.
When you use urllib2
import urllib2 as urlget
The desired output of the code will be that of the url http://stackoverflow.com/search?q=hellojai
>>>url = "http://stackoverflow.com/search"
>>> data = {'q':"hellojai"}
>>> data = urllib.urlencode(data)
>>> data = urlget.urlopen(url,data)
see for the status code.
data.code
data.getcode()
data.message
FOR headers
1.data.headers.headers
2.data.url
For the Requests it wil like this
query = {'q':"hellojai"}
>>> datadata = requests.get(url,params=query)
Note : params is required.
This is how we make get request.
For post in both
import urllib
import urllib2
import webbrowser
url = "http://duckduckgo.com/html"
data = urllib.urlencode({'q': 'Python'})
results = urllib2.urlopen(url, data)
with open("results.html", "w") as f:
f.write(results.read())
webbrowser.open("results.html")
For REQUESTS (requests)
import requests
url = "http://duckduckgo.com/html"
payload = {'q':'python'}
r = requests.post(url, payload)
with open("requests_results.html", "w") as f:
f.write(r.content)
Further read about the Request and urlopen in using urllib2.
The class "Request" you're asking about: http://docs.python.org/library/urllib2.html#urllib2.Request
class urllib2.Request(url[, data][, headers][, origin_req_host][, unverifiable])
This class is an abstraction of a URL request.
The function you actually want to make a request (which can accept aRequest
object or wrap one around a URL string you provice) constructing a Request object):http://docs.python.org/library/urllib2.html#urllib2.urlopen
urllib2.urlopen(url[, data][,timeout]) Open the URL url, which can be either a string or a Request object.
Example:theurl = "www.example.com"
try:
resp = urllib2.urlopen(theurl)
print resp.read()
except IOError as e:
print "Error: ", eExample 2 (withRequest
):theurl = "www.example.com"
try:
req = urllib2.Request(theurl)
print req.get_full_url()
print req.get_method()
print dir(req) # list lots of other stuff in Request
resp = urllib2.urlopen(req)
print resp.read()
except IOError as e:
print "Error: ", e