Package org.wikidata.wdtk.util

Class WebResourceFetcherImpl

java.lang.Object
org.wikidata.wdtk.util.WebResourceFetcherImpl
All Implemented Interfaces:
WebResourceFetcher
public class WebResourceFetcherImpl extends Object implements WebResourceFetcher
Standard implementation of WebResourceFetcher.
Author:
Markus Kroetzsch

  • Field Details

    • userAgent

      protected static String userAgent

    • proxy

      protected static Proxy proxy

  • Constructor Details

    • WebResourceFetcherImpl

      public WebResourceFetcherImpl()

  • Method Details

    • getProxy

      public static Proxy getProxy()
      Returns the proxy that will be used for all requests made by Wikidata Toolkit.
      Returns:
      the proxy represented as java object

    • setProxy

      public static void setProxy(Proxy proxy)
      Sets the proxy that will be used for alle requests made by Wikidata Toolkit. This should be set in own tools based on Wikidata Toolkit esp. when making large amounts of requests.
      Parameters:
      proxy - the proxy represented as java object

    • hasProxy

      public static boolean hasProxy()
      Checks whether a proxy is set.
      Returns:
      True if a proxy is set, false, if there isn't set any proxy.

    • getUserAgent

      public static String getUserAgent()
      Returns the string that will be used to identify the user agent on all requests made by Wikidata Toolkit.
      Returns:
      the user agent string

    • setUserAgent

      public static void setUserAgent(String userAgent)
      Sets the string that will be used to identify the user agent on all requests made by Wikidata Toolkit. This should be set in own tools based on Wikidata Toolkit esp. when making large amounts of requests.
      Parameters:
      userAgent - the user agent string

    • getUrlConnection

      public static URLConnection getUrlConnection(URL url) throws IOException
      Opens a basic URL connection for the given URL and performs basic configurations. In particular, it will set the User-Agent. The current proxy settings are also respected. For http(s) URLs, the result is a HttpURLConnection.
      Parameters:
      url - the URL to open
      Returns:
      the URL connection to access this URL
      Throws:
      IOException

    • getInputStreamForUrl

      public InputStream getInputStreamForUrl(String urlString) throws IOException
      Description copied from interface: WebResourceFetcher
      Returns an InputStream for the document at the given URL. This can be used for downloading. The stream should be closed after use.
      Specified by:
      getInputStreamForUrl in interface WebResourceFetcher
      Parameters:
      urlString - the URL of the document
      Returns:
      InputStream for the requested document
      Throws:
      IOException - if the document at the URL could not be opened or the URL was invalid