Hitachi Vantara Pentaho Community Forums
Results 1 to 14 of 14

Thread: How to set HTTP Request Headers in HTTP Client Step

  1. #1
    Join Date
    May 2009
    Posts
    6

    Default How to set HTTP Request Headers in HTTP Client Step

    Hi all,

    Is there a way to control the request headers in the HTTP Client step?

    Best regards,
    RS94202

  2. #2
    Join Date
    May 2006
    Posts
    4,882

    Default

    Not yet.

    Regards,
    Sven

  3. #3
    Join Date
    May 2009
    Posts
    6

    Default

    Hi Sven,

    But I still need to send HTTP requests w/ controlling the headres, what would be the recommended alternative? Could I do it through Javascript somehow?

    Best regards,
    RS94202

  4. #4
    Join Date
    May 2006
    Posts
    4,882

    Default

    Doubtful... best routes would probably to:

    a) Hack yourself a change to http step... If it's useful for other people make the code available, else probably the best is to move the changed http step to your own step
    b) Request a change via JIRA.

    Or depending on what you really need:
    - Script what you need outside of PDI, e.g. with curl or so.

    Regards,
    Sven

  5. #5
    Join Date
    May 2009
    Posts
    6

    Default

    It might be best to proceed w/ a), any guidance how to do this? get source, build locally, then upload changes? as a attachment somewhere or? is there a page that describes the process?

  6. #6
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Request a change via JIRA.
    If you have code to contribute, attach it.
    We'll commit the changes.

  7. #7
    Join Date
    May 2009
    Posts
    6

    Default

    Sorry guys, I needed it urgently - I did it with Java Script Step using the Apache HTTP Client lib. I am pasting the code in case somebody else needs a quick solution for accessing the HTTP Request Headers:

    var httpclient = new Packages.org.apache.http.impl.client.DefaultHttpClient();
    var httpget = new Packages.org.apache.http.client.methods.HttpGet(theURL);
    //httpget.setHeader("Accept-Language", "en-us,en;q=0.5");
    //httpget.setHeader("Accept-Encoding", "gzip,deflate");
    httpget.setHeader("Accept-Charset","ISO-8859-1,utf-8;q=0.7,*;q=0.7");
    var responseHandler = new Packages.org.apache.http.impl.client.BasicResponseHandler();
    var resultBody = httpclient.execute(httpget, responseHandler);
    httpclient.getConnectionManager().shutdown();

    that is the code in the step and one should also bring the Apache HTTP Client jars (this sample code compiles against 4.0-beta2.tar.gz "Binary with dependencies" from http://hc.apache.org/downloads.cgi)

    Best regards,
    RS94202

  8. #8
    Join Date
    Jul 2009
    Posts
    17

    Default

    Hi
    Actually there is no need to include a jar file as commons-httpclient-3.0.1.jar is already in included with Pentaho Data Integration in libext/commons.

    Here is some sample code on how you could use this in the Modified Javascript Value Step and make a HTTP GET request with a custom header:

    var httpclient = new Packages.org.apache.commons.httpclient.HttpClient();
    var method = new Packages.org.apache.commons.httpclient.methods.GetMethod("http://www.google.com");

    method.addRequestHeader("Your Header Name","Your Header Value");

    var http_status_code = httpclient.executeMethod( method );
    var http_status_text = method.getStatusText();
    var http_response = method.getResponseBodyAsString();
    method.releaseConnection();

  9. #9
    Join Date
    Jul 2009
    Posts
    17

    Default

    Quote Originally Posted by MattCasters View Post
    Request a change via JIRA.
    If you have code to contribute, attach it.
    We'll commit the changes.
    I have raised this request on JIRA and a patch file is also attached. See PDI-4295:

    http://jira.pentaho.com/browse/PDI-4295

  10. #10
    Join Date
    Jun 2009
    Posts
    3

    Default

    Just wanted to add a further note for anyone who needs to set headers AND use authentication together. Hope this helps someone.

    Code:
    //Adapted from the Basic Auth example in the Apache Commonds HTTPClient examples with mfantcook's addRequestHeader thrown in the mix
    client = new Packages.org.apache.commons.httpclient.HttpClient;
    usrpsw = new Packages.org.apache.commons.httpclient.UsernamePasswordCredentials("myReallyLongSuperSecureApiTokenThingy", "myReallyLongSuperSecureApiTokenThingy");
    authsc = new Packages.org.apache.commons.httpclient.auth.AuthScope("mysubdomain.mysaasapp.com", 80, "My Realm - Use CURL to discover");
    // create a new GET method using basic authentication from above.
    method = new Packages.org.apache.commons.httpclient.methods.GetMethod("http://mysubdomain.mysaasapp.com/myresource.xml");
    
    // pass our credentials to HttpClient
    client.getState().setCredentials(authsc,usrpsw);
    
    // Tell the GET method to automatically handle authentication.
    method.setDoAuthentication( true );
    
    // Add an "If-None-Match" header to pass an ETag for change checking 
    // This can MASSIVELY reduce wire transfer if the host supports it
    // Note the embedded double quotes in the etag string, this is essential.
    method.addRequestHeader("If-None-Match",'"4e6993f454dc19b5246cceb38376b546"');
    
    // Add an "If-Modified-Since" header to pass an ETag for change checking 
    // This can also reduce wire transfer but carries the added burden of time zones, etc.
    method.addRequestHeader("If-Modified-Since","Thu, 26 Aug 2010 08:23:02 GMT");
    
    // Add "Accept" and "Content-Type" headers, may be necessary for some APIs.
    method.addRequestHeader("Accept","application/xml");
    method.addRequestHeader("Content-Type","application/xml");
    
    var status = client.executeMethod( method );
    var message = method.getStatusText();
    var response = method.getResponseBodyAsString();
    var request = method.getRequestHeaders().toSource()
    //var query = method.getQueryString(); 
    //var path = method.getPath();
    
    method.releaseConnection();
    Note to Matt and the PDI devs in general: Thanks for the awesome tool! The HTTP Client is actually really useful for most situations but when I need to consume RESTful web services it's a bit too simple. It would be perfect if we could specify the HTTP verb (GET/POST/etc) and set arbitrary headers.
    Last edited by joeharris76; 08-28-2010 at 04:50 AM. Reason: Formatting

  11. #11

    Default

    Just a quick note to say that v4.1.0 supports arbitrary headers and authentication in HTTP Client.

    In Kettle, there are 2 separate steps for GET and POST which has been fine for the REST web services I've used. I was wondering if the other REST methods (PUT/DELETE) are actually used by any REST services in practice?
    Tony, Cloud2Land.com

    Get the latest snapshot Pentaho Data Integration (Kettle) builds from SVN on my blog!
    http://www.cloud2land.com/

  12. #12
    Join Date
    Jun 2009
    Posts
    3

    Default

    Quote Originally Posted by Tony.Cloud2Land View Post
    J …I was wondering if the other REST methods (PUT/DELETE) are actually used by any REST services in practice?
    The would be a huge YES; if they don't then it's not really a restful API.

    See for example: Basecamp, Highrise, FreeAgent, etc. Pretty much any site that has been built on Rails will expect to use all 4 verbs.


    Joe

  13. #13
    Join Date
    Sep 2011
    Posts
    1

    Default

    thanks Joe Harris.. this helped me a lot..

  14. #14
    Join Date
    Mar 2006
    Posts
    170

    Default

    Hi All,

    Just curious as I have not dived into the Web Service capabilities of PDI but I know others who need to use "ws-security".

    Does the work-arounds in this thread provide a method to satisfy the needs of using services that require "ws-security" in the interaction?

    Thanks

    Kent

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.