REST

This chapter looks at recipes around REST web services, via Lift’s RestHelper trait. For an introduction, take a look at the Lift wiki page and Chapter 5 of Simply Lift.

The sample code from this chapter is at https://github.com/LiftCookbook/cookbook_rest.

DRY URLs

Problem

You find yourself repeating parts of URL paths in your RestHelper and you Don’t want to Repeat Yourself (DRY).

Solution

Use prefix in your RestHelper:

package code.rest

import net.liftweb.http.rest.RestHelper
import net.liftweb.http.LiftRules

object IssuesService extends RestHelper {

  def init() : Unit = {
    LiftRules.statelessDispatch.append(IssuesService)
  }

  serve("issues" / "by-state" prefix {
    case "open" :: Nil XmlGet _ => <p>None open</p>
    case "closed" :: Nil XmlGet _ => <p>None closed</p>
    case "closed" :: Nil XmlDelete _ => <p>All deleted</p>
  })
}

This service responds to URLs of /issues/by-state/open and /issues/by-state/closed and we have factored out the common part as a prefix.

Wire this into Boot.scala with:

import code.rest.IssuesService
IssuesService.init()

We can test the service with cURL:

$ curl -H 'Content-Type: application/xml'
    http://localhost:8080/issues/by-state/open
<?xml version="1.0" encoding="UTF-8"?>
<p>None open</p>

$ curl -X DELETE -H 'Content-Type: application/xml'
    http://localhost:8080/issues/by-state/closed
<?xml version="1.0" encoding="UTF-8"?>
<p>All deleted</p>

Discussion

You can have many serve blocks in your RestHelper, which helps give your REST service structure.

In this example, we’ve arbitrarily decided to return XML and to match on an XML request using XmlGet and XmlDelete. The test for an XML request requires a content type of text/xml or application/xml, or a request for a path that ends with .xml. This is why the cURL request includes a header with with the -H flag. If we hadn’t included that, the request would not match any of our patterns, and the result would be a 404 response.

Missing File Suffix

Problem

Your RestHelper expects a filename as part of the URL, but the suffix (extension) is missing, and you need it.

Solution

Access req.path.suffix to recover the suffix.

For example, when processing /download/123.png, you want to be able to reconstruct 123.png:

package code.rest

import net.liftweb.http.rest.RestHelper
import net.liftweb.http.LiftRules
import xml.Text

object Reunite extends RestHelper {

  private def reunite(name: String, suffix: String) =
    if (suffix.isEmpty) name else name+"."+suffix

  serve {
    case "download" :: file :: Nil Get req =>
      Text("You requested "+reunite(file, req.path.suffix))
  }

  def init() : Unit = {
    LiftRules.statelessDispatch.append(Reunite)
  }

}

We are matching on download but rather than using the file value directly, we pass it through the reunite function first to attach the suffix back on (if any).

Requesting this URL with a command like cURL will show you the filename as expected:

$ curl http://127.0.0.1:8080/download/123.png
<?xml version="1.0" encoding="UTF-8"?>
You requested 123.png

Discussion

When Lift parses a request, it splits the request into constituent parts (e.g., turning the path into a List[String]). This includes a separation of some suffixes. This is good for pattern matching when you want to change behaviour based on the suffix, but a hindrance in this particular situation.

Only those suffixes defined in LiftRules.explicitlyParsedSuffixes are split from the filename. This includes many of the common file suffixes (such as .png, .atom, .json) and also some you may not be so familiar with, such as .com.

Note that if the suffix is not in explicitlyParsedSuffixes, the suffix will be an empty string and the name (in the previous example) will be the filename with the suffix still attached.

Depending on your needs, you could alternatively add a guard condition to check for the file suffix:

case "download" :: file :: Nil Get req if req.path.suffix == "png" =>
  Text("You requested PNG file called "+file)

Or rather than simply attaching the suffix back on, you could take the opportunity to do some computation and decide what to send back. For example, if the client supports the WebP image format, you might prefer to send that:

package code.rest

import net.liftweb.http.rest.RestHelper
import net.liftweb.http.LiftRules
import xml.Text

object Reunite extends RestHelper  {

  def init() : Unit = {
    LiftRules.statelessDispatch.append(Reunite)
  }

  serve {
    case "negotiate" :: file :: Nil Get req =>
      val toSend =
        if (req.header("Accept").exists(_ == "image/webp")) file+".webp"
        else file+".png"

      Text("You requested "+file+", would send "+toSend)
  }

}

Calling this service would check the HTTP Accept header before deciding what resource to send:

$ curl http://localhost:8080/negotiate/123
<?xml version="1.0" encoding="UTF-8"?>
You requested 123, would send 123.png

$ curl http://localhost:8080/negotiate/123 -H "Accept: image/webp"
<?xml version="1.0" encoding="UTF-8"?>
You requested 123, would send 123.webp

Missing .com from Email Addresses

When submitting an email address to a REST service, a domain ending .com is stripped before your REST service can handle the request.

Solution

Modify LiftRules.explicitlyParsedSuffixes so that Lift doesn’t change URLs that end with .com.

In Boot.scala:

import net.liftweb.util.Helpers
LiftRules.explicitlyParsedSuffixes = Helpers.knownSuffixes - "com"

Discussion

By default, Lift will strip off file suffixes from URLs to make it easy to match on suffixes. An example would be needing to match on all requests ending in .xml or .pdf. However, .com is also registered as one of those suffixes, but this is inconvenient if you have URLs that end with email addresses.

Note that this doesn’t impact email addresses in the middle of URLs. For example, consider the following REST service:

package code.rest

import net.liftweb.http.rest.RestHelper
import net.liftweb.http.LiftRules
import xml.Text

object Suffix extends RestHelper {

  def init() : Unit = {
    LiftRules.statelessDispatch.append(Suffix)
  }

  serve {
    case "email" :: e :: "send" :: Nil Get req =>
      Text("In middle: "+e)

    case "email" :: e :: Nil Get req =>
      Text("At end: "+e)
  }

}

With this service init method called in Boot.scala, we could then make requests and observe the issue:

$ curl http://localhost:8080/email/you@example.com/send
<?xml version="1.0" encoding="UTF-8"?>
In middle: you@example.com

$ curl http://localhost:8080/email/you@example.com
<?xml version="1.0" encoding="UTF-8"?>
At end: you@example

The .com is being treated as a file suffix, which is why the solution of removing it from the list of suffixes will resolve this problem.

Note that because other top-level domains, such as .uk, .nl, .gov, are not in explicitlyParsedSuffixes, those email addresses are left untouched.

Failing to Match on a File Suffix

Problem

You’re trying to match on a file suffix (extension), but your match is failing.

Solution

Ensure the suffix you’re matching on is included in LiftRules.explicitlyParsedSuffixes.

As an example, perhaps you want to match anything ending in .csv at your /reports/ URL:

case Req("reports" :: name :: Nil, "csv", GetRequest) =>
  Text("Here's your CSV report for "+name)

You’re expecting /reports/foo.csv to produce "Here’s your CSV report for foo," but you get a 404.

To resolve this, include "csv" as a file suffix that Lift knows to split from URLs. In Boot.scala, call:

LiftRules.explicitlyParsedSuffixes += "csv"

The pattern will now match.

Discussion

Without adding "csv" to the explicitlyParsedSuffixes, the example URL would match with:

case Req("reports" :: name :: Nil, "", GetRequest) =>
  Text("Here's your CSV report for "+name)

Here we’re matching on no suffix (""). In this case, name would be set to "foo.csv". This is because Lift separates file suffixes from the end of URLs only for file suffixes that are registered with explicitlyParsedSuffixes. Because "csv" is not one of the default registered suffixes, "foo.csv" is not split. That’s why "csv" in the suffix position of Req pattern match won’t match the request, but an empty string in that position will.

Accept Binary Data in a REST Service

Problem

You want to accept an image upload, or other binary data, in your RESTful service.

Solution

Access the request body in your REST helper:

package code.rest

import net.liftweb.http.rest.RestHelper
import net.liftweb.http.LiftRules

object Upload extends RestHelper {

  def init() : Unit = {
    LiftRules.statelessDispatch.append(Upload)
  }

  serve {
    case "upload" :: Nil Post req =>
      for {
        bodyBytes <- req.body
      } yield <info>Received {bodyBytes.length} bytes</info>
  }

}

Wire this into your application in Boot.scala:

import code.rest.Upload
Upload.init()

Test this service using a tool like cURL:

$ curl -X POST --data-binary "@dog.jpg" -H 'Content-Type: image/jpg'
    http://127.0.0.1:8080/upload
<?xml version="1.0" encoding="UTF-8"?>
<info>Received 1365418 bytes</info>

Discussion

In this example, the binary data is accessed via the req.body, which returns a Box[Array[Byte]]. We turn this into a Box[Elem] to send back to the client. Implicits in RestHelper turn this into an XmlResponse for Lift to handle.

Note that web containers, such as Jetty and Tomcat, may place limits on the size of an upload. You will recognise this situation by an error such as java.lang.IllegalStateException: Form too large705784>200000. Check with documentation for the container for changing these limits.

To restrict the type of image you accept, you could add a guard condition to the match, but you may find you have more readable code by moving the logic into an unapply method on an object. For example, to restrict an upload to just a JPEG you could say:

serve {
  case "jpg" :: Nil Post JPeg(req) =>
    for {
      bodyBytes <- req.body
    } yield <info>Jpeg Received {bodyBytes.length} bytes</info>
  }

object JPeg {
  def unapply(req: Req): Option[Req] =
    req.contentType.filter(_ == "image/jpg").map(_ => req)
}

We have defined an extractor called JPeg that returns Some[Req] if the content type of the upload is image/jpg; otherwise, the result will be None. This is used in the REST pattern match as JPeg(req). Note that the unapply needs to return Option[Req] as this is what’s expected by the Post extractor.

Serving Videos

Problem

You want to use the HTML5 video tag to serve videos.

Solution

Use a RestHelper to return a chunked StreamingResponse of a video.

A short example:

import net.liftweb.http.StreamingResponse
import net.liftweb.http.rest.RestHelper

object Streamer extends RestHelper {
 serve {
    case req@Req(("video" :: id :: Nil), _, _) =>
      val fis = new FileInputStream(new File(id))
      val size = file.length - 1
      val content_type = "Content-Type" -> "video/mp4" // replace with appropriate format

      val start = 0L
      val end = size

      val headers =
        ("Connection" -> "close") ::
        ("Transfer-Encoding" -> "chunked") ::
        content_type ::
        ("Content-Range" -> ("bytes " + start.toString + "-" + end.toString + "/" + file.length.toString)) ::
        Nil

      () => StreamingResponse(
        data = fis,
        onEnd = fis.close,
        size,
        headers,
        cookies = Nil,
        code = 206
    )
  }
}

Discussion

The above example doesn’t allow the client to skip ahead in the file. The Range header must be parsed in order to facilitate this. It specifies what part of the file the client wishes to receive. Request headers are contained within the req@Req(urlParams, _, _) pattern in the previous example. The start and end can be extracted like so:

val (start,end) =
  req.header("Range") match {
    case Full(r) => {
      (
         parseNumber(r.substring(r.indexOf("bytes=") + 6)),
         {
           if (r.endsWith("-"))
             file.length - 1
           else
             parseNumber(r.substring(r.indexOf("-") + 1))
         }
       )
    }
    case _ => (0L, file.length - 1)
  }

Next, the response must skip ahead to the location in the video where the client wishes to begin. This is done by doing the following:

val fis: FileInputStream = ... // Shown in the first example
fis.skip(start)

Returning JSON

Problem

You want to return JSON from a REST call.

Solution

Use the Lift JSON domain-specific language (DSL). For example:

package code.rest

import net.liftweb.http.rest.RestHelper
import net.liftweb.http.LiftRules
import net.liftweb.json.JsonAST._
import net.liftweb.json.JsonDSL._

object QuotationsAPI extends RestHelper {

  def init() : Unit = {
    LiftRules.statelessDispatch.append(QuotationsAPI)
  }

  serve {
    case "quotation" :: Nil JsonGet req =>
     ("text" -> "A beach house isn't just real estate. It's a state of mind.") ~
     ("by" -> "Douglas Adams") : JValue
  }

}

Wire this into Boot.scala:

import code.rest.QuotationsAPI
QuotationsAPI.init()

Running this example produces:

$ curl -H 'Content-type: text/json' http://127.0.0.1:8080/quotation
{
 "text":"A beach house isn't just real estate. It's a state of mind.",
 "by":"Douglas Adams"
}

Discussion

The type ascription at the end of the JSON expression (: JValue) tells the compiler that the expression is expected to be of type JValue. This is required to allow the DSL to apply. It would not be required if, for example, you were calling a function that was defined to return a JValue.

The JSON DSL allows you to create nested structures, lists, and everything else you expect of JSON.

In addition to the DSL, you can create JSON from a case class by using the Extraction.decompose method:

import net.liftweb.json.Extraction
import net.liftweb.json.DefaultFormats

case class Quote(text: String, by: String)
val quote = Quote(
  "A beach house isn't just real estate. It's a state of mind.",
  "Douglas Adams")

implicit val formats = DefaultFormats
val json : JValue = Extraction decompose quote

This will also produce a JValue, which when printed will be:

{
 "text":"A beach house isn't just real estate. It's a state of mind.",
 "by":"Douglas Adams"
}

Google Sitemap

Problem

You want to make a Google Sitemap using Lift’s rendering capabilities.

Solution

Create the Google site map structure, and bind to it as you would for any template in Lift. Then deliver it as XML content.

Start with a sitemap.html in your webapp folder containing valid XML-Sitemap markup:

<?xml version="1.0" encoding="utf-8" ?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    <url data-lift="SitemapContent.base">
        <loc></loc>
        <changefreq>daily</changefreq>
        <priority>1.0</priority>
        <lastmod></lastmod>
    </url>
    <url data-lift="SitemapContent.list">
        <loc></loc>
        <lastmod></lastmod>
    </url>
</urlset>

Make a snippet to fill the required gaps:

package code.snippet

import org.joda.time.DateTime
import net.liftweb.util.CssSel
import net.liftweb.http.S
import net.liftweb.util.Helpers._

class SitemapContent {

  case class Post(url: String, date: DateTime)

  lazy val entries =
    Post("/welcome", new DateTime) :: Post("/about", new DateTime) :: Nil

  val siteLastUdated = new DateTime

  def base: CssSel =
    "loc *" #> "http://%s/".format(S.hostName) &
      "lastmod *" #> siteLastUpdated.toString("yyyy-MM-dd'T'HH:mm:ss.SSSZZ")

  def list: CssSel =
    "url *" #> entries.map(post =>
      "loc *" #> "http://%s%s".format(S.hostName, post.url) &
        "lastmod *" #> post.date.toString("yyyy-MM-dd'T'HH:mm:ss.SSSZZ"))

}

This example is using canned data for two pages.

Apply the template and snippet in a REST service at /sitemap:

package code.rest

import net.liftweb.http._
import net.liftweb.http.rest.RestHelper

object Sitemap extends RestHelper {
  serve {
    case "sitemap" :: Nil Get req =>
      XmlResponse(
        S.render(<lift:embed what="sitemap" />, req.request).head
      )
  }
}

Wire this into your application in Boot.scala, and force Lift to process /sitemap as XML:

LiftRules.statelessDispatch.append(code.rest.Sitemap)

LiftRules.htmlProperties.default.set({ request: Req =>
  request.path.partPath match {
    case "sitemap" :: Nil => OldHtmlProperties(request.userAgent)
    case _                => Html5Properties(request.userAgent)
  }
})

Test this service using a tool like cURL:

$ curl http://127.0.0.1:8080/sitemap

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>http://127.0.0.1/</loc>
      <changefreq>daily</changefreq>
      <priority>1.0</priority>
      <lastmod>2013-02-10T19:16:12.433+00:00</lastmod>
  </url>
  <url>
      <loc>http://127.0.0.1/welcome</loc>
      <lastmod>2013-02-10T19:16:12.434+00:00</lastmod>
  </url><url>
      <loc>http://127.0.0.1/about</loc>
      <lastmod>2013-02-10T19:16:12.434+00:00</lastmod>
  </url>
</urlset>

Discussion

You may be wondering why we’ve used REST here when we could have used a regular HTML template and snippet. The reason is that we want XML rather than HTML output. We use the same mechanism, but invoke it and wrap it in an XmlResponse.

Using Lift’s regular snippet mechanism for rendering this XML is convenient. But although we are working with XML, Lift will be parsing sitemap.html using the default HTML5 parser. The behaviour of the parser follows the HTML5 specification and this is not the same as the behaviour you might expect from a parser for XML. For example, recognized HTML tags in an invalid position are moved by the HTML parser. To avoid these situations we have modified Boot.scala to use an XML parser for /sitemap.

The S.render method takes a NodeSeq and an HTTPRequest. The first we supply by running the sitemap.html snippet; the second is simply the current request. XmlResponse requires a Node rather than a NodeSeq, which is why we call head—as there’s only one node in the response, this does what we need.

Note that Google Sitemaps needs dates to be in ISO 8601 format. The built-in java.text.SimpleDateFormat does not support this format prior to Java 7. If you are using Java 6, you need to use org.joda.time.DateTime as we are in this example.

Calling REST Service from a Native iOS Application

Problem

You want to make an HTTP POST from a native iOS device to a Lift REST service.

Solution

Use NSURLConnection, ensuring you set the content-type to application/json.

For example, suppose we want to call this service:

package code.rest

import net.liftweb.http.rest.RestHelper
import net.liftweb.json.JsonDSL._
import net.liftweb.json.JsonAST._

object Shouty extends RestHelper {

  def greet(name: String) : JValue =
    "greeting" -> ("HELLO "+name.toUpperCase)

  serve {
    case "shout" :: Nil JsonPost json->request =>
      for { JString(name) <- (json \\ "name").toOpt }
      yield greet(name)
  }

}

The service expects a JSON post with a parameter of name, and it returns a greeting as a JSON object. To demonstrate the data to and from the service, we can include the service in Boot.scala:

LiftRules.statelessDispatch.append(Shouty)

Call it from the command line:

$ curl -d '{ "name" : "World" }' -X POST -H 'Content-type: application/json'
   http://127.0.0.1:8080/shout
{
  "greeting":"HELLO WORLD"
}

We can implement the POST request using NSURLConnection:

static NSString *url = @"http://localhost:8080/shout";

-(void) postAction {
  // JSON data:
  NSDictionary* dic = @{@"name": @"World"};
  NSData* jsonData =
    [NSJSONSerialization dataWithJSONObject:dic options:0 error:nil];
  NSMutableURLRequest *request = [
    NSMutableURLRequest requestWithURL:[NSURL URLWithString:url]
    cachePolicy:NSURLRequestUseProtocolCachePolicy timeoutInterval:60.0];

  // Construct HTTP request:
  [request setHTTPMethod:@"POST"];
  [request setValue:@"application/json" forHTTPHeaderField:@"Content-Type"];
  [request setValue:[NSString stringWithFormat:@"%d", [jsonData length]]
    forHTTPHeaderField:@"Content-Length"];
  [request setHTTPBody: jsonData];

  // Send the request:
  NSURLConnection *con = [[NSURLConnection alloc]
    initWithRequest:request delegate:self];
}

- (void)connection:(NSURLConnection *)connection
  didReceiveResponse:(NSURLResponse *)response {
   // Start off with new, empty, response data
   self.receivedJSONData = [NSMutableData data];
}

- (void)connection:(NSURLConnection *)connection
  didReceiveData:(NSData *)data {
   // append incoming data
   [self.receivedJSONData appendData:data];
}

- (void)connection:(NSURLConnection *)connection
  didFailWithError:(NSError *)error {
   NSLog(@"Error occurred ");
}

- (void)connectionDidFinishLoading:(NSURLConnection *)connection {
  NSError *e = nil;
  NSDictionary *JSON =
    [NSJSONSerialization JSONObjectWithData: self.receivedJSONData
    options: NSJSONReadingMutableContainers error: &e];
  NSLog(@"Return result: %@", [JSON objectForKey:@"greeting"]);
}

Obviously in this example we’ve used hardcoded values and URLs, but this will hopefully be a starting point for use in your application.

Discussion

There are many ways to do HTTP POST from iOS, and it can be confusing to identify the correct way that works, especially without the aid of an external library. The previous example uses the iOS native API.

Another way is to use AFNetworking. This is a popular external library for iOS development, can cope with many scenarios, and is simple to use:

#import "AFHTTPClient.h"
#import "AFNetworking.h"
#import "JSONKit.h"

static NSString *url = @"http://localhost:8080/shout";

-(void) postAction {
  // JSON data:
  NSDictionary* dic = @{@"name": @"World"};
  NSData* jsonData =
   [NSJSONSerialization dataWithJSONObject:dic options:0 error:nil];

  // Construct HTTP request:
  NSMutableURLRequest *request =
   [NSMutableURLRequest requestWithURL:[NSURL URLWithString:url]
    cachePolicy:NSURLRequestUseProtocolCachePolicy timeoutInterval:60.0];
  [request setHTTPMethod:@"POST"];
  [request setValue:@"application/json" forHTTPHeaderField:@"Content-Type"];
  [request setValue:[NSString stringWithFormat:@"%d", [jsonData length]]
    forHTTPHeaderField:@"Content-Length"];
  [request setHTTPBody: jsonData];

  // Send the request:
  AFJSONRequestOperation *operation =
    [[AFJSONRequestOperation alloc] initWithRequest: request];
  [operation setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation,
    id responseObject)
  {
     NSString *response = [operation responseString];

     // Use JSONKit to deserialize the response into NSDictionary
     NSDictionary *deserializedJSON = [response objectFromJSONString];
     [deserializedJSON count];

     // The response object can be a NSDicionary or a NSArray:
      if([deserializedJSON count]> 0) {
         NSLog(@"Return value: %@",[deserializedJSON objectForKey:@"greeting"]);
      }
      else {
        NSArray *deserializedJSONArray = [response objectFromJSONString];
        NSLog(@"Return array value: %@", deserializedJSONArray );
      }
  }failure:^(AFHTTPRequestOperation *operation, NSError *error)
  {
    NSLog(@"Error: %@",error);
  }];
  [operation start];
}

The NSURLConnection approach is more versatile and gives you a starting point to craft your own solution, such as by making the content type more specific. However, AFNetworking is popular and you may prefer that route.

Files

04-REST.asciidoc

Latest commit

History

04-REST.asciidoc

File metadata and controls

REST

DRY URLs

Problem

Solution

Discussion

See Also

Missing File Suffix

Problem

Solution

Discussion

See Also

Missing .com from Email Addresses

Solution

Discussion

See Also

Failing to Match on a File Suffix

Problem

Solution

Discussion

See Also

Accept Binary Data in a REST Service

Problem

Solution

Discussion

See Also

Serving Videos

Problem

Solution

Discussion

See Also

Returning JSON

Problem

Solution

Discussion

See Also

Google Sitemap

Problem

Solution

Discussion

See Also

Calling REST Service from a Native iOS Application

Problem

Solution

Discussion

See Also