edu.uci.ics.crawler4j.robotstxt

Class RobotstxtParser

java.lang.Object

edu.uci.ics.crawler4j.robotstxt.RobotstxtParser

public class RobotstxtParser
extends Object

Author:

Yasser Ganjisaffar

Constructor Summary

Constructors

Constructor and Description
RobotstxtParser()

Method Summary

Methods

Modifier and Type	Method and Description
static HostDirectives	parse(String content, String myUserAgent)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RobotstxtParser

public RobotstxtParser()

Method Detail

parse

public static HostDirectives parse(String content,
                   String myUserAgent)