edu.uci.ics.crawler4j.crawler

Class Page

java.lang.Object

edu.uci.ics.crawler4j.crawler.Page

public class Page
extends Object

This class contains the data for a fetched and parsed page.

Author:

Yasser Ganjisaffar

Field Summary

Fields

Modifier and Type	Field and Description
protected String	contentCharset The charset of the content.
protected byte[]	contentData The content of this page in binary format.
protected String	contentEncoding The encoding of the content.
protected String	contentType The ContentType of this page.
protected org.apache.http.Header[]	fetchResponseHeaders Headers which were present in the response of the fetch request
protected ParseData	parseData The parsed data populated by parsers
protected WebURL	url The URL of this page.

Constructor Summary

Constructors

Constructor and Description
Page(WebURL url)

Method Summary

Methods

Modifier and Type	Method and Description
String	getContentCharset() Returns the charset of the content.
byte[]	getContentData() Returns the content of this page in binary format.
String	getContentEncoding() Returns the encoding of the content.
String	getContentType() Returns the ContentType of this page.
org.apache.http.Header[]	getFetchResponseHeaders() Returns headers which were present in the response of the fetch request
ParseData	getParseData() Returns the parsed data generated for this page by parsers
WebURL	getWebURL()
void	load(org.apache.http.HttpEntity entity) Loads the content of this page from a fetched HttpEntity.
void	setContentCharset(String contentCharset)
void	setContentData(byte[] contentData)
void	setContentEncoding(String contentEncoding)
void	setContentType(String contentType)
void	setFetchResponseHeaders(org.apache.http.Header[] headers)
void	setParseData(ParseData parseData)
void	setWebURL(WebURL url)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

url

protected WebURL url

The URL of this page.

contentData

protected byte[] contentData

The content of this page in binary format.

contentType

protected String contentType

The ContentType of this page. For example: "text/html; charset=UTF-8"

contentEncoding

protected String contentEncoding

The encoding of the content. For example: "gzip"

contentCharset

protected String contentCharset

The charset of the content. For example: "UTF-8"

fetchResponseHeaders

protected org.apache.http.Header[] fetchResponseHeaders

Headers which were present in the response of the fetch request

parseData

protected ParseData parseData

The parsed data populated by parsers

Constructor Detail

Page

public Page(WebURL url)

Method Detail

getWebURL

public WebURL getWebURL()

setWebURL

public void setWebURL(WebURL url)

load

public void load(org.apache.http.HttpEntity entity)
          throws Exception

Loads the content of this page from a fetched HttpEntity.

Throws:

Exception

getFetchResponseHeaders

public org.apache.http.Header[] getFetchResponseHeaders()

Returns headers which were present in the response of the fetch request

setFetchResponseHeaders

public void setFetchResponseHeaders(org.apache.http.Header[] headers)

getParseData

public ParseData getParseData()

Returns the parsed data generated for this page by parsers

setParseData

public void setParseData(ParseData parseData)

getContentData

public byte[] getContentData()

Returns the content of this page in binary format.

setContentData

public void setContentData(byte[] contentData)

getContentType

public String getContentType()

Returns the ContentType of this page. For example: "text/html; charset=UTF-8"

setContentType

public void setContentType(String contentType)

getContentEncoding

public String getContentEncoding()

Returns the encoding of the content. For example: "gzip"

setContentEncoding

public void setContentEncoding(String contentEncoding)

getContentCharset

public String getContentCharset()

Returns the charset of the content. For example: "UTF-8"

setContentCharset

public void setContentCharset(String contentCharset)