org.dice_research.squirrel.queue

Interface IpAddressBasedQueue

All Superinterfaces:

UriQueue

All Known Implementing Classes:

AbstractIpAddressBasedQueue, InMemoryQueue

public interface IpAddressBasedQueue
extends UriQueue

This extension of the UriQueue interface defines additional methods enabling the queue to manage the retrieving of chunks of URIs based on IP addresses. If a chunk is returned by this queue, the IP addresses are marked as blocked. No other chunk will contain URIs of these IP addresses until the method markIpAddressAsAccessible(InetAddress) is called to free the IP address.

Author:

Michael Röder (roeder@informatik.uni-leipzig.de)

Method Summary

Modifier and Type	Method and Description
Iterator<AbstractMap.SimpleEntry<InetAddress,List<CrawleableUri>>>	getIPURIIterator() Goes through the queue und collects all IP-address with their URIs
int	getNumberOfBlockedIps() Returns the number of IP addresses that are currently blocked.
void	markIpAddressAsAccessible(InetAddress ip) Marks the given IP address as accessible.

Methods inherited from interface org.dice_research.squirrel.queue.UriQueue

addUri, close, getNextUris, isEmpty, open

Method Detail

markIpAddressAsAccessible

void markIpAddressAsAccessible(InetAddress ip)

Marks the given IP address as accessible.

Parameters:

ip - the IP address that should be marked as accessible.

getNumberOfBlockedIps

int getNumberOfBlockedIps()

Returns the number of IP addresses that are currently blocked.

Returns:

the number of IP addresses that are currently blocked.

getIPURIIterator

Iterator<AbstractMap.SimpleEntry<InetAddress,List<CrawleableUri>>> getIPURIIterator()

Goes through the queue und collects all IP-address with their URIs

Returns:

a IP-address-iterator with the list of uris for each IP-address