ASPNETWebStack Wiki Rss Feed

Updated Wiki: Web API Request Batching

yaohuang — Mon, 13 May 2013 17:28:58 GMT

Overview

Request batching is a useful way of minimizing the number of messages that are passed between the client and the server. This reduces network traffic and provides a smoother, less chatty user interface. This feature will enable Web API users to batch multiple HTTP requests and send them as a single HTTP request.

Scenarios

To enable batching in general, we’re providing custom message handlers (i.e. DefaultHttpBatchHandler, DefaultODataBatchHandler) which you can register per-route to handle the batch requests.

Web API Batching

Registering HTTP batch endpoint

You can use MapHttpBatchRoute, which is an HttpRouteCollection extension method, to create a batch endpoint. For example, the following will create a batch endpoint at “api/$batch”.

using System.Web.Http;
using System.Web.Http.Batch;

namespace BatchSample
{
    public static class WebApiConfig
    {
        public static void Register(HttpConfiguration config)
        {
            config.Routes.MapHttpBatchRoute(
                routeName: "WebApiBatch", 
                routeTemplate: "api/$batch",
                batchHandler: new DefaultHttpBatchHandler(GlobalConfiguration.DefaultServer));

            config.Routes.MapHttpRoute(
                name: "DefaultApi",
                routeTemplate: "api/{controller}/{id}",
                defaults: new { id = RouteParameter.Optional }
            );
        }
    }
}

That’s all you need to do on the server side. Now, on the client side you can use the existing Web API client library to submit a batch request .

using System.Net.Http;
using System.Net.Http.Formatting;

namespace BatchClientSample
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            string baseAddress = "http://localhost:8080";
            HttpClient client = new HttpClient();
            HttpRequestMessage batchRequest = new HttpRequestMessage(HttpMethod.Post, baseAddress + "/api/$batch")
            {
                Content = new MultipartContent("mixed")
                {
                    // POST http://localhost:8080/api/values
                    new HttpMessageContent(new HttpRequestMessage(HttpMethod.Post, baseAddress + "/api/values")
                    {
                        Content = new ObjectContent<string>("my value", new JsonMediaTypeFormatter())
                    }),

                    // GET http://localhost:8080/api/values
                    new HttpMessageContent(new HttpRequestMessage(HttpMethod.Get, baseAddress + "/api/values"))
                }
            };

            HttpResponseMessage batchResponse = client.SendAsync(batchRequest).Result;

            MultipartStreamProvider streamProvider = batchResponse.Content.ReadAsMultipartAsync().Result;
            foreach (var content in streamProvider.Contents)
            {
                HttpResponseMessage response = content.ReadAsHttpResponseMessageAsync().Result;

                // Do something with the response messages
            }
        }
    }
}

Changing the ExecutionOrder of the DefaultHttpBatchHandler

By default each individual batch request is executed sequentially. Meaning the second request in the batch won’t start until the first one is completed. If you have a scenario where the order of execution is not important and you want to execute the requests asynchronously, you can set the ExecutionOrder property on the DefaultHttpBatchHandler to NonSequential.

using System.Web.Http;
using System.Web.Http.Batch;

namespace BatchSample
{
    public static class WebApiConfig
    {
        public static void Register(HttpConfiguration config)
        {
            HttpBatchHandler batchHandler = new DefaultHttpBatchHandler(GlobalConfiguration.DefaultServer)
            {
                ExecutionOrder = BatchExecutionOrder.NonSequential
            };

            config.Routes.MapHttpBatchRoute(
                routeName: "WebApiBatch",
                routeTemplate: "api/$batch",
                batchHandler: batchHandler);

            config.Routes.MapHttpRoute(
                name: "DefaultApi",
                routeTemplate: "api/{controller}/{id}",
                defaults: new { id = RouteParameter.Optional }
            );
        }
    }
}

OData Batching

Registering OData batch endpoint

You can simply pass an ODataBatchHandler to the MapODataRoute to enable the batching. The batch endpoint will be available at routePrefix/$batch. For instance if you have the following OData route, the batch endpoint will be exposed at “odata/$batch”.

using System.Web.Http;
using System.Web.Http.OData.Batch;
using System.Web.Http.OData.Builder;
using BatchODataSample.Controllers;
using Microsoft.Data.Edm;

namespace BatchODataSample
{
    public static class WebApiConfig
    {
        public static void Register(HttpConfiguration config)
        {
            config.Routes.MapODataRoute(
                routeName: "defaultOdata",
                routePrefix: "odata",
                model: GetModel(),
                batchHandler: new DefaultODataBatchHandler(GlobalConfiguration.DefaultServer));
        }

        private static IEdmModel GetModel()
        {
            ODataConventionModelBuilder builder = new ODataConventionModelBuilder();
            builder.Namespace = "BatchODataSample.Controllers";
            builder.EntitySet<Customer>("Customers");
            builder.EntitySet<Order>("Orders");
            return builder.GetEdmModel();
        }
    }
}

That’s it. Now for the client, you can use WCF Data Services Client Library:

using System;
using System.Data.Services.Client;
using BatchClientSample.ServiceReference1;

namespace BatchClientSample
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            string baseAddress = "http://localhost:8080/odata";
            Container container = new Container(new Uri(baseAddress));

            int id = new Random().Next();
            var customer = new Customer { ID = id, Name = "User" + id };
            var order = new Order { ID = id, Amount = id + 10 };

            // Batch operation.
            container.AddToCustomers(customer);
            container.AddToOrders(order);
            container.AddLink(customer, "Orders", order);

            var batchResponse = container.SaveChanges(SaveChangesOptions.Batch);

            foreach (var response in batchResponse)
            {
                Console.WriteLine(response.StatusCode);
                Console.WriteLine(response.Headers);
            }
        }
    }
}

Or datajs (or any JavaScript library that supports sending OData batch requests):

OData.request({
    requestUri: "/odata/$batch",
    method: "POST",
    data: {
        __batchRequests: [
            { __changeRequests: [
                { requestUri: "Customers", method: "POST", data: customer }
            ] },
            { requestUri: "Customers", method: "GET" }
        ]
    }
}, function (data, response) {
    //success handler
}, function () {
    alert("request failed");
}, OData.batchHandler);

Setting up multiple OData routes and the batch endpoints

You can have multiple OData routes with their own batch endpoints. For example, the following will setup the batch endpoints at “catalog/$batch” and “commerce/$batch”.

public static void Register(HttpConfiguration config)
{
    config.Routes.MapODataRoute(
        routeName: "odata1",
        routePrefix: "catalog",
        model: GetModel(),
        batchHandler: new DefaultODataBatchHandler(GlobalConfiguration.DefaultServer));

    config.Routes.MapODataRoute(
        routeName: "odata2",
        routePrefix: "commerce",
        model: GetModel2(),
        batchHandler: new DefaultODataBatchHandler(GlobalConfiguration.DefaultServer));
}

Note that the ODataBatchHandler instance cannot be shared across routes if you want to support relative URI in the OData batch requests. For example, the following request constructed using datajs will submit a batch request to “/commerce/$batch” but the requestUri for the sub-requests are simply “Customers”. In this case the ODataBatchHandler need to be aware of the OData route where it’s registered to figure out the right route prefix (in this case “commerce”) for the requestUri.

OData.request({
    requestUri: "/commerce/$batch",
    method: "POST",
    data: {
        __batchRequests: [
            { __changeRequests: [
                { requestUri: "Customers", method: "POST", data: customer }
            ] },
            { requestUri: "Customers", method: "GET" }
        ]
    }
}, function (data, response) {
    //success handler
}, function () {
    alert("request failed");
}, OData.batchHandler);

Setting Batch Quotas

You can set the throttle on the Batch by setting the MessageQuotas on the ODataBatchHandler. For instance, the following setting will only allow a maximum of 10 requests per batch and 10 operations per ChangeSet.

using System.Web.Http;
using System.Web.Http.OData.Batch;
using System.Web.Http.OData.Builder;
using BatchODataSample.Controllers;
using Microsoft.Data.Edm;

namespace BatchODataSample
{
    public static class WebApiConfig
    {
        public static void Register(HttpConfiguration config)
        {
            ODataBatchHandler odataBatchHandler = new DefaultODataBatchHandler(GlobalConfiguration.DefaultServer);
            odataBatchHandler.MessageQuotas.MaxOperationsPerChangeset = 10;
            odataBatchHandler.MessageQuotas.MaxPartsPerBatch = 10;

            config.Routes.MapODataRoute(
                routeName: "defaultOdata",
                routePrefix: "odata",
                model: GetModel(),
                batchHandler: odataBatchHandler);
        }

        private static IEdmModel GetModel()
        {
            ODataConventionModelBuilder builder = new ODataConventionModelBuilder();
            builder.Namespace = "BatchODataSample.Controllers";
            builder.EntitySet<Customer>("Customers");
            builder.EntitySet<Order>("Orders");
            return builder.GetEdmModel();
        }
    }
}

Custom Batching

You can derive from either HttpBatchHandler or DefaultHttpBatchHandler to support custom batch formats. For instance, instead of using MIME multipart, you can use JSON as the format for the batch requests just like Facebook batch requests.

Here is a naïve implementation of HttpBatchHandler that encodes the batch requests/responses as JSON. It simply derives from DefaultHttpBatchHandler, overrides the ParseBatchRequestsAsync/CreateResponseMessageAsync methods and let the base class handle the rest.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Threading.Tasks;
using System.Web.Http;
using System.Web.Http.Batch;

namespace BatchSample
{
    public class JsonBatchHandler : DefaultHttpBatchHandler
    {
        public JsonBatchHandler(HttpServer server)
            : base(server)
        {
            SupportedContentTypes.Add("text/json");
            SupportedContentTypes.Add("application/json");
        }

        public override async Task<IList<HttpRequestMessage>> ParseBatchRequestsAsync(HttpRequestMessage request)
        {
            var jsonSubRequests = await request.Content.ReadAsAsync<JsonRequestMessage[]>();

            // Creating simple requests, no headers nor bodies
            var subRequests = jsonSubRequests.Select(r =>
            {
                Uri subRequestUri = new Uri(request.RequestUri, "/" + r.relative_url);
                return new HttpRequestMessage(new HttpMethod(r.method), subRequestUri);
            });
            return subRequests.ToList();
        }

        public override async Task<HttpResponseMessage> CreateResponseMessageAsync(IList<HttpResponseMessage> responses,
                                                                                   HttpRequestMessage request)
        {
            List<JsonResponseMessage> jsonResponses = new List<JsonResponseMessage>();
            foreach (var subResponse in responses)
            {
                var jsonResponse = new JsonResponseMessage
                {
                    code = (int)subResponse.StatusCode
                };
                foreach (var header in subResponse.Headers)
                {
                    jsonResponse.headers.Add(header.Key, String.Join(",", header.Value));
                }
                if (subResponse.Content != null)
                {
                    jsonResponse.body = await subResponse.Content.ReadAsStringAsync();
                    foreach (var header in subResponse.Content.Headers)
                    {
                        jsonResponse.headers.Add(header.Key, String.Join(",", header.Value));
                    }
                }
                jsonResponses.Add(jsonResponse);
            }

            return request.CreateResponse<List<JsonResponseMessage>>(HttpStatusCode.OK, jsonResponses);
        }
    }

    public class JsonResponseMessage
    {
        public JsonResponseMessage()
        {
            headers = new Dictionary<string, string>();
        }

        public int code { get; set; }

        public Dictionary<string, string> headers { get; set; }

        public string body { get; set; }
    }

    public class JsonRequestMessage
    {
        public string method { get; set; }

        public string relative_url { get; set; }
    }
}

Just like with any other HttpBatchHandler, you can register the batch endpoint using MapHttpBatchRoute.

using System.Web.Http;
using System.Web.Http.Batch;
using BatchSample;

namespace BatchRequestSample
{
    public static class WebApiConfig
    {
        public static void Register(HttpConfiguration config)
        {
            config.Routes.MapHttpBatchRoute(
                routeName: "WebApiBatch",
                routeTemplate: "api/$batch",
                batchHandler: new DefaultHttpBatchHandler(GlobalConfiguration.DefaultServer));

            config.Routes.MapHttpBatchRoute(
                routeName: "WebApiBatchJson",
                routeTemplate: "api/$batchJson",
                batchHandler: new JsonBatchHandler(GlobalConfiguration.DefaultServer));

            config.Routes.MapHttpRoute(
                name: "DefaultApi",
                routeTemplate: "api/{controller}/{id}",
                defaults: new { id = RouteParameter.Optional }
            );
        }
    }
}

Design

HttpBatchHandler

This is a custom HttpMessageHandler that is used to handle the batch requests. The HttpBatchHandler takes an HttpServer in the constructor and use it to dispatch the sub-requests.

HttpBatchHandler is an abstract class and an implementation of HttpBatchHandler will typically do the following:

Parse the incoming request into sub-requests
Execute the batch requests
Build the batch response

Here is a high level overview of how HttpBatchHandler interacts with other handlers/dispatchers in the Web API pipeline. Note that the HttpBatchHandler is registered as a per-route handler as you’ve seen in the sample code above.

HttpBatchHandler Implementations

Out of the box, we provide different HttpBatchHandler implementations to support simple Web API batching as well as OData batching. Here is an overview of the HttpBatchHandler hierarchy.

DefaultHttpBatchHandler

This is a simple batch handler that encodes the HTTP request/response messages as MIME multipart. By default, it buffers the HTTP request messages in-memory during parsing. The DefaultHttpBatchHandler has several virtual methods which you can use to extend and customize.

ODataHttpBatchHandler

For OData batching we provide two implementations:

DefaultODataBatchHandler – Supports OData batch formats, sub-requests are sent once all the sub-requests are read, it buffers the content stream of the sub-requests.
UnbufferedODataBatchHandler – Supports OData batch formats, individual sub-request is sent as soon as it is read, it doesn’t buffer the content stream of the sub-requests.

Why can’t I set ExecutionOrder in ODataHttpBatchHandler like in DefaultHttpBatchHandler?

The OData spec has the execution order defined and it could be problematic if changed because the client can make certain assumptions based on the spec. By definition, the operation/ChangeSet within a batch request is executed in ordered manner. Although the operations within the ChangeSet can be executed regardless of the order but our implementation will execute them sequentially for simplicity (easier to deal with Content-ID references).

Content-ID references

We do support Content-ID header which is a mechanism used for referencing requests in a ChangeSet. Here is the description from the OData spec: “If a MIME part representing an Insert request within a ChangeSet includes a Content-ID header, then the new entity may be referenced by subsequent requests within the same ChangeSet by referring to the Content-ID value prefixed with a "$" character. When used in this way, $<contentIdValue> acts as an alias for the Resource Path that identifies the new entity.”

The way how we implement this is by building a Content-ID to Location header dictionary when processing the ChangeSet. After sending each request in the ChangeSet, we build the dictionary using the Content-ID header from the request and the Location header from the response. And before sending each request in the ChangeSet, we use the dictionary to replace the $<contentIdValue> in the request URI with the Location.

Raw Batch Formats

DefaultHttpBatchHandler

Sample Request

POST http://localhost:8080/api/$batch HTTP/1.1
Content-Type: multipart/mixed; boundary="91731eeb-d443-4aa6-9816-560a8aca66b1"
Host: localhost:8080
Content-Length: 390
Expect: 100-continue
Connection: Keep-Alive

--91731eeb-d443-4aa6-9816-560a8aca66b1
Content-Type: application/http; msgtype=request

POST /api/values HTTP/1.1
Host: localhost:8080
Content-Type: application/json; charset=utf-8

"my value"
--91731eeb-d443-4aa6-9816-560a8aca66b1
Content-Type: application/http; msgtype=request

GET /api/values HTTP/1.1
Host: localhost:8080


--91731eeb-d443-4aa6-9816-560a8aca66b1--

Sample Response

HTTP/1.1 200 OK
Content-Length: 333
Content-Type: multipart/mixed; boundary="5b2a806d-4040-43f0-8f04-7d4c86793fa7"
Server: Microsoft-HTTPAPI/2.0
Date: Mon, 08 Apr 2013 19:00:26 GMT

--5b2a806d-4040-43f0-8f04-7d4c86793fa7
Content-Type: application/http; msgtype=response

HTTP/1.1 202 Accepted


--5b2a806d-4040-43f0-8f04-7d4c86793fa7
Content-Type: application/http; msgtype=response

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

["my value"]
--5b2a806d-4040-43f0-8f04-7d4c86793fa7--

DefaultODataBatchHandler/UnbufferedODataBatchHandler

Sample Request

POST /service/$batch HTTP/1.1 
Host: host 
Content-Type: multipart/mixed; boundary=batch_36522ad7-fc75-4b56-8c71-56071383e77b 

--batch_36522ad7-fc75-4b56-8c71-56071383e77b 
Content-Type: multipart/mixed; boundary=changeset_77162fcd-b8da-41ac-a9f8-9357efbbd621 
Content-Length: ###       

--changeset(77162fcd-b8da-41ac-a9f8-9357efbbd621) 
Content-Type: application/http 
Content-Transfer-Encoding: binary 
Content-ID: 1 

POST /service/Customers HTTP/1.1 
Host: host  
Content-Type: application/atom+xml;type=entry 
Content-Length: ### 

<AtomPub representation of a new Customer> 

--changeset(77162fcd-b8da-41ac-a9f8-9357efbbd621) 
Content-Type: application/http 
Content-Transfer-Encoding: binary 

POST $1/Orders HTTP/1.1 
Host: host 
Content-Type: application/atom+xml;type=entry 
Content-Length: ### 

<AtomPub representation of a new Order> 

--changeset(77162fcd-b8da-41ac-a9f8-9357efbbd621)-- 
--batch(36522ad7-fc75-4b56-8c71-56071383e77b)--

Sample Response

HTTP/1.1 202 Accepted
DataServiceVersion: 1.0
Content-Length: ####
Content-Type: multipart/mixed; boundary=batch(36522ad7-fc75-4b56-8c71-56071383e77b)

--batch(36522ad7-fc75-4b56-8c71-56071383e77b)
Content-Type: application/http
Content-Transfer-Encoding: binary

HTTP/1.1 200 Ok
Content-Type: application/atom+xml;type=entry
Content-Length: ###

<AtomPub representation of the Customer entity with EntityKey ALFKI>

--batch(36522ad7-fc75-4b56-8c71-56071383e77b)
Content-Type: multipart/mixed; boundary=changeset(77162fcd-b8da-41ac-a9f8-9357efbbd621)
Content-Length: ###      

--changeset(77162fcd-b8da-41ac-a9f8-9357efbbd621)
Content-Type: application/http
Content-Transfer-Encoding: binary

HTTP/1.1 201 Created
Content-Type: application/atom+xml;type=entry
Location: http://host/service.svc/Customer('POIUY')
Content-Length: ###

<AtomPub representation of a new Customer entity>

--changeset(77162fcd-b8da-41ac-a9f8-9357efbbd621)
Content-Type: application/http
Content-Transfer-Encoding: binary

HTTP/1.1 204 No Content
Host: host

--changeset(77162fcd-b8da-41ac-a9f8-9357efbbd621)--

--batch(36522ad7-fc75-4b56-8c71-56071383e77b)
Content-Type: application/http
Content-Transfer-Encoding: binary

HTTP/1.1 404 Not Found
Content-Type: application/xml
Content-Length: ###

<Error message>
--batch(36522ad7-fc75-4b56-8c71-56071383e77b)--

New Comment on "$select and $expand support"

RomanBats — Sun, 12 May 2013 09:50:08 GMT

Expand doesn't work if I try to get entity by key? http://server/Customers(1)?$expand=Orders

New Comment on "CORS support for ASP.NET Web API"

amarta78 — Sat, 11 May 2013 00:35:53 GMT

I have got the answer for this ....check Yao's blog for the answer: http://blogs.msdn.com/b/yaohuang1/archive/2013/04/05/try-out-asp.net-web-api-cors-support-using-the-nightly-builds.aspx thanks a lot!

New Comment on "CORS support for ASP.NET Web API"

amarta78 — Fri, 10 May 2013 01:04:14 GMT

I get a compiler error saying that EnableCorsAttribute does not contain a constructor that takes 0 arguments. Can you please let us know why we get this?

Updated Wiki: $select and $expand support

raghuramn — Thu, 09 May 2013 21:04:33 GMT

$select and $expand support

The $select system query option allows clients to requests a limited set of information for each entity or complex type identified by the ResourcePath and other System Query Options like $filter, $top, $skip etc. The $select query option is often used in conjunction with the $expand query option, to first increase the scope of the resource graph returned ($expand) and then selectively prune that resource graph ($select).

The $expand system query option allows clients to request related resources when a resource that satisfies a particular request is retrieved.

The following model would be used for the rest of the scenarios,

public class Customer
{
    public int ID { get; set; }

    public string Name { get; set; }

    public virtual ICollection<Order> Orders { get; set; }
}

public class SpecialCustomer : Customer
{
    public int SpecialCustomerProperty { get; set; }

    public virtual ICollection<SpecialOrder> SpecialOrders { get; set; }
}

public class Order
{
    public int ID { get; set; }

    public int Amount { get; set; }

    public Customer Customer { get; set; }
}

public class SpecialOrder : Order
{
    public int SpecialOrderProperty { get; set; }

    public virtual SpecialCustomer SpecialCustomer { get; set; }
}

private static IEdmModel GetModel()
{
    ODataConventionModelBuilder builder = new ODataConventionModelBuilder();
    builder.EntitySet<Customer>("Customers");
    builder.EntitySet<Order>("Orders");
    return builder.GetEdmModel();
}

For example, assuming that a service exposes two entity sets ‘Customers’ and ‘Orders’ and a customer has 1..* (one-to-many) relationship with order, the following URL’s are valid,

Supported scenarios

http://server/Customers?$select=Id,Name
Response should contain only the properties Id and Name for each customer. Navigation link for the property Orders must not be present in the response.

http://server/Customers?$expand=Orders
Response should contain all the properties of Customer for each customer. It should also contain the expanded Orders for each customer along with the navigation link (depending on the metadata level though). The expanded Orders should contain all structural properties of Orders and navigation link for each navigation property on Order.

http://server/Customers?$select=Id,Name,Orders&$expand=Orders
Response should contain properties Id and Name for each customer and expanded Orders. Each expanded Order should contain all the structural properties of order and a navigation link for each navigation property of order.

http://server/Customers?$select=Orders/Id&$expand=Orders
Selection on an expanded property.

http://server/Customers?$select=Id,NS.SpecialCustomer/SpecialId
Response should contain Id property for each customer and also includes the SpecialId property for each special customer. ODataLib uri parser doesn’t support this yet.

http://server/Customers?$expand=NS.SpecialCustomer/SpecialOrder
Response should contain all structural properties and expand the SpecialOrder property for each special customer. A normal customer would have his navigation links as deferred content in the response. ODataLib uri parser doesn’t support this yet.

Not supported scenarios

http://server/Customers(42)/Address?$select=City
Selections of complex types is not supported. (OData V4 might have support for it though)

Samples

1) Supporting $select and $expand on collections through QueryableAttribute.

[Queryable]
public IQueryable<Customer> GetCustomers()
{
    return _dbContext.Customers;
}

2) Supporting $select and $expand on single entities through QueryableAttribute.

[Queryable]
public SingleResult<Customer> GetCustomer(int id)
{
    return SingleResult.Create(_dbContext.Customers.Where(c => c.ID == id);
}

3) Supporting $select and $expand on single entities when you don't have a db context (Linq2Objects).

[Queryable]
public Customer GetCustomer(int id)
{
    return _repository.Find(id);
}

4) Supporting $select and $expand with ODataQueryOptions<T>

This is more complex than usual as Request.CreateResponse doesn't have a non generic implementation. There is only Request.CreateResponse<T>. This is beign tracked with this bug

public class CustomersController : ODataController
{
    private CustomersContext _db = new CustomersContext();

    public HttpResponseMessage GetCustomers(ODataQueryOptions<Customer> query)
    {
        return CreateQueryableResponse(Request, query.ApplyTo(_db.Customers));
    }

    private static HttpResponseMessage CreateQueryableResponse(HttpRequestMessage request, IQueryable queryable)
    {
        MethodInfo mi = typeof(CustomersController).GetMethod("CreateQueryableResponseImpl", BindingFlags.Static | BindingFlags.NonPublic);
        mi = mi.MakeGenericMethod(queryable.ElementType);
        return mi.Invoke(null, new object[] { request, queryable }) as HttpResponseMessage;
    }

    private static HttpResponseMessage CreateQueryableResponseImpl<T>(HttpRequestMessage request, IQueryable<T> response)
    {
        return request.CreateResponse(response as IQueryable<T>);
    }
}

5) Supporting only select.

[Queryable(AllowedQueryOptions = AllowedQueryOptions.Select)]
public IQueryable<Customer> GetCustomers()
{
    return _dbContext.Customers;
}

6) Controlling maximum expansion depth.

[Queryable(MaxExpansionDepth = 1)]
public IQueryable<Customer> GetCustomers()
{
    return _dbContext.Customers;
}

7) Controlling allowed expandable properties.

public class CustomSelectExpandQueryValidator : SelectExpandQueryValidator
{
    private HashSet<string> _allowedExpandedProperties;

    public CustomSelectExpandQueryValidator(params string[] allowedExpandedProperties)
    {
        _allowedExpandedProperties = new HashSet<string>(allowedExpandedProperties);
    }

    public override void Validate(SelectExpandQueryOption selectExpandQueryOption, ODataValidationSettings validationSettings)
    {
        base.Validate(selectExpandQueryOption, validationSettings);

        SelectExpandClause clause = selectExpandQueryOption.SelectExpandClause;
        foreach (ExpandItem item in clause.Expansion.ExpandItems)
        {
            NavigationPropertySegment navigationPropertySegment = item.PathToNavigationProperty.LastOrDefault() as NavigationPropertySegment;
            if (navigationPropertySegment != null)
            {
                string propertyName = navigationPropertySegment.NavigationProperty.Name;
                if (!_allowedExpandedProperties.Contains(propertyName))
                {
                    throw new ODataException(String.Format("Navigation property '{0}' cannot be expanded.", propertyName));
                }
            }
        }
    }
}

Considerations and Open questions

There are several open questions that have to be discussed and concluded.

Support for other formatters

Should we support $select with json and xml formatters? $expand is not important as these formatters expand by default.
Should we change the behavior of these formatters to not expand by default if the incoming query contains a $expand clause?

Query optimizations

1. Should $select be optimized?

$select is a shape changing query. From a web API perspective, this would mean that the ObjectContent<T> has to be reconstructed. What would happen to the content headers on the ObjectContent? Should we just do a content-negotiation again and say that any content-type headers set after the action finished and before the QueryableAttribute runs would be lost?
An un-optimized $select would mean more memory allocation on the database.

design meeting@Mar 28^th: First cut, actions returning IQueryable will support $select and $expand as it won't be a shape changing query anymore. $select will be optimized.

2. $expand has to be optimized.

Without an optimized $expand no-one would ever use our $expand support unless they are building on top of in-memory IQueryable.
How would the generated query look like? Should we just support EF or try be compatible with some other providers? (Scoping question).
If we do it only for EF, what is the extensibility story for custom providers?

design meeting@Mar 28^th: work item: update with the queries

3. $select and $expand can be done on a single entity. In these cases there is no IQueryable returned by the action. How can we do query manipulation in this scenario? For example, http://server/Customers(42)?$select=Name maps to an web API action
public Customer GetCustomer(int key).

PageSize for expanded feeds.

How can one specify page size for expanded feeds? WCF DS has a per entity-set configuration knob for page size. Should we follow their model?

Implementation details

The idea here is that a selection ($select) results in a LINQ .Select query using a wrapper class (kind of an static anonymous type) as the response. The wrapper class contains the properties that have to be included in the selection. For example, a $select involving a selection having one property uses a wrapper class that looks like this,

public class SelectExpandWrapper<TElement> 
{
    public PropertyContainer Container { get; set; }
}
internal abstract class PropertyContainer
{
        public Dictionary<string, object> ToDictionary(bool includeAutoSelected = true)
}

Also, we wanted to free the OData formatter from having to look at the $select and $expand options. To support that, the wrapper class should contain information about three things,

what structural properties/navigation properties have to be serialized.
what actions have to be included in the response.
what navigation properties have to be expanded.

To support that the wrapper should also implement the IEdmStructuredObject interface.

public interface IEdmStructuredObject : IEdmObject
{
        bool TryGetValue(string propertyName, out object value);
}

public interface IEdmObject
{
        IEdmTypeReference GetEdmType();
}

The IEdmObject interface is just a way to get the EDM type name required for odata serialization and also provides a dynamic property accessor to separate the implementation of the Wrapper from the formatter. Users who don't like our Wrapper implementation (geared towards EF and Linq2Objects) could consider doing their own implementations.

Generated queries

There are two things to consider here.

OData formatter (in the default case), requires the key properties to generate navigation links. So, even if the client did not ask for key properties explicitly in the request, they have to be fetched from the database. We should probably have an extension point here, so that people who customize link generation can fetch extra properties if they want them for link generation.
OData formatter requires the type name of the entity while writing responses. If we are getting a partial object from the database (wrapper class), we have to include the type name in the wrapper class.

I am listing out the possible generated LINQ queries here for some scenarios for reference. The generated SQL queries (using EF) would be too complex in some cases. If you really want to look at them, refer to the following dump files,

using Table per hierarchy strategy (TPH) which is code first default - tph_dump.txt
using table per type strategy (TPT) - tpt_dump.txt

Also, if you are interested in playing around with the sample, I have the cs file here.

simple $select

uri = "~/Customers?$select=ID";
query = customers.Select<Customer, Wrapper>(c => new Wrapper<int>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    Name1 = "ID",
    Value1 = c.ID
});

Response:

{
  "__type": "NS.Customer",
  "ID": 1
}
{
  "__type": "NS.SpecialCustomer",
  "ID": 2
}

simple $expand

uri = "~/Customers?$expand=Orders";
query = customers.Select(c => new SelectExpandWrapper<Customer>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    Instance = c,
    PropertyContainer = new NamedProperty<Order>
    {
        Name = "Orders",
        Value = c.Orders.Select(o => new SelectExpandWrapper
        {
            TypeName = o is SpecialOrder ? typeNameSpecialOrder : typeNameOrder,
            Instance = o
        })
    }
});

Response:

{
  "__type": "NS.Customer",
  "ID": 1,
  "Name": "Raghu",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 1,
      "Amount": 400
    }
  ]
}
{
  "__type": "NS.SpecialCustomer",
  "ID": 2,
  "Name": "Ram",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 2,
      "Amount": 1000
    }
  ]
}

simple $select and $expand

uri = "~/Customers?$select=Name,Orders&$expand=Orders";
query = customers.Select(c => new SelectExpandWrapper<Customer>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    PropertyContainer = new NamedProperty<string> 
    {
        Name = "Name",
        Value = c.Name,
        Next = new NamedProperty<IEnumerable<Order>>
        {
            Name = "Orders",
            Value = c.Orders.Select<Order, Wrapper>(o => new Wrapper
            {
                TypeName = o is SpecialOrder ? typeNameSpecialOrder : typeNameOrder
                Instance = o
            }
        })
    }
});

Response:
{
  "__type": "NS.Customer",
  "Name": "Raghu",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 1,
      "Amount": 400
    }
  ]
}
{
  "__type": "NS.SpecialCustomer",
  "Name": "Ram",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 2,
      "Amount": 1000
    }
  ]
}

selecting properties from expanded results

uri = "~/Customers?$select=Name,Orders/ID,Orders/Amount&$expand=Orders";
query = customers.Select(c => new SelectExpandWrapper<Customer>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    PropertyContainer = new NamedProperty<string>
    {
        Name = "Name",
        Value = c.Name,
        Next = new NamedProperty<IEnumerable<SelectExpandWrapper<Order>>>
        {
            Name = "Orders",
            Value = c.Orders(o => new SelectExpandWrapper<Order>
            {
                TypeName = o is SpecialOrder ? typeNameSpecialOrder : typeNameOrder,
                PropertyContainer = new NamedProperty<int>
                {
                    Name = "ID",
                    Value = o.ID,
                    Next = new NamedProperty<int> 
                    {
                        Name = "Amount",
                        Value = o.Amount
                    }
                }
            }
    })
});

Response:

{
  "__type": "NS.Customer",
  "Name": "Raghu",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 1,
      "Amount": 400
    }
  ]
}
{
  "__type": "NS.SpecialCustomer",
  "Name": "Ram",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 2,
      "Amount": 1000
    }
  ]
}

expanding properties on derived types

uri = "~/Customers?$expand=NS.SpecialCustomer/SpecialOrders";
query = customers.Select(c => new SelectExpandWrapper<Customer>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    Instance = c,
    PropertyContainer = new NamedProperty<SpecialCustomer>
    {
        Name = c is SpecialCustomer ? "SpecialOrders" : null,
        Value = (c as SpecialCustomer).Orders.Select(o => new SelectExpandWrapper<Order>
        {
            TypeName = o is SpecialOrder ? typeNameSpecialOrder : typeNameOrder,
            Instance = o
        }) //requires null propagation. also don't confuse null and non-existence
    }
});

Response:

{
  "__type": "NS.Customer",
  "ID": 1,
  "Name": "Raghu"
}
{
  "__type": "NS.SpecialCustomer",
  "ID": 2,
  "Name": "Ram",
  "SpecialOrders": [
    {
      "__type": "NS.Order",
      "ID": 2,
      "Amount": 1000
    }
  ]
}

selecting properties on derived types

uri = "~/Customers?$select=Name,NS.SpecialCustomer/SpecialCustomerProperty";
query = customers.Select(c => new SelectExpandWrapper<Customer>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    PropertyContainer = new NamedProperty<string>
    {
        Name = "Name",
        Value = c.Name,
        Next = new PropertyContainer<SpecialCustomer>
        {
            Name = c is SpecialCustomer ? "SpecialCustomerProperty" : null,
            Value = (c as SpecialCustomer).SpecialCustomerProperty // requires null propagation.
        }
    }
});

Response:

{
  "__type": "NS.Customer",
  "Name": "Raghu"
}
{
  "__type": "NS.SpecialCustomer",
  "Name": "Ram",
  "SpecialCustomerProperty": 42
}

expanding properties on expanded derived types

uri = "~/Customers?$select=Name&$expand=Orders,Orders/NS.SpecialOrder/SpecialCustomer";
query = customers.Select<Customer, Wrapper>(c => new SelectExpandWrapper<Customer>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    PropertyContainer = new NamedProperty<string>
    {
        Name = "Name",
        Value = c.Name,
        Next = new NamedProperty<IEnumerable<SelectExpandWrapper<Order>>>
        {
            Name = "Orders",
            Value = c.Orders.Select(o => new SelectExpandWrapper<Order>
            {
                Instance = o,
                Name = o is SpecialOrder ? "SpecialCustomer" : null, // optional property
                Value = new Wrapper
                {
                    Instance = (o as SpecialOrder).SpecialCustomer
                }
            }
        }
    })
});

Response:

{
  "__type": "NS.Customer",
  "Name": "Raghu",
  "Orders": [
    {
      "__type": null,
      "ID": 1,
      "Amount": 400
    }
  ]
}
{
  "__type": "NS.SpecialCustomer",
  "Name": "Ram",
  "Orders": [
    {
      "__type": null,
      "ID": 2,
      "Amount": 1000,
      "SpecialCustomer": {
        "__type": "NS.SpecialCustomer",
        "ID": 2,
        "Name": "Ram",
        "SpecialCustomerProperty": 42
      }
    }
  ]
}

Updated Wiki: $select and $expand support

raghuramn — Thu, 09 May 2013 20:46:52 GMT

$select and $expand support

public class Customer
{
    public int ID { get; set; }

    public string Name { get; set; }

    public virtual ICollection<Order> Orders { get; set; }
}

public class SpecialCustomer : Customer
{
    public int SpecialCustomerProperty { get; set; }

    public virtual ICollection<SpecialOrder> SpecialOrders { get; set; }
}

public class Order
{
    public int ID { get; set; }

    public int Amount { get; set; }

    public Customer Customer { get; set; }
}

public class SpecialOrder : Order
{
    public int SpecialOrderProperty { get; set; }

    public virtual SpecialCustomer SpecialCustomer { get; set; }
}

private static IEdmModel GetModel()
{
    ODataConventionModelBuilder builder = new ODataConventionModelBuilder();
    builder.EntitySet<Customer>("Customers");
    builder.EntitySet<Order>("Orders");
    return builder.GetEdmModel();
}

For example, assuming that a service exposes two entity sets ‘Customers’ and ‘Orders’ and a customer has 1..* (one-to-many) relationship with order, the following URL’s are valid,

Supported scenarios

Not supported scenarios

http://server/Customers(42)/Address?$select=City
Selections of complex types is not supported. (OData V4 might have support for it though)

Samples

1) Supporting $select and $expand on collections through QueryableAttribute.

[Queryable]
public IQueryable<Customer> GetCustomers()
{
    return _dbContext.Customers;
}

2) Supporting $select and $expand on single entities through QueryableAttribute.

[Queryable]
public SingleResult<Customer> GetCustomer(int id)
{
    return SingleResult.Create(_dbContext.Customers.Where(c => c.ID == id);
}

3) Supporting $select and $expand on single entities when you don't have a db context (Linq2Objects).

[Queryable]
public Customer GetCustomer(int id)
{
    return _repository.Find(id);
}

4) Supporting only select.

[Queryable(AllowedQueryOptions = AllowedQueryOptions.Select)]
public IQueryable<Customer> GetCustomers()
{
    return _dbContext.Customers;
}

5) Controlling maximum expansion depth.

[Queryable(MaxExpansionDepth = 1)]
public IQueryable<Customer> GetCustomers()
{
    return _dbContext.Customers;
}

6) Controlling allowed expandable properties.

public class CustomSelectExpandQueryValidator : SelectExpandQueryValidator
{
    private HashSet<string> _allowedExpandedProperties;

    public CustomSelectExpandQueryValidator(params string[] allowedExpandedProperties)
    {
        _allowedExpandedProperties = new HashSet<string>(allowedExpandedProperties);
    }

    public override void Validate(SelectExpandQueryOption selectExpandQueryOption, ODataValidationSettings validationSettings)
    {
        base.Validate(selectExpandQueryOption, validationSettings);

        SelectExpandClause clause = selectExpandQueryOption.SelectExpandClause;
        foreach (ExpandItem item in clause.Expansion.ExpandItems)
        {
            NavigationPropertySegment navigationPropertySegment = item.PathToNavigationProperty.LastOrDefault() as NavigationPropertySegment;
            if (navigationPropertySegment != null)
            {
                string propertyName = navigationPropertySegment.NavigationProperty.Name;
                if (!_allowedExpandedProperties.Contains(propertyName))
                {
                    throw new ODataException(String.Format("Navigation property '{0}' cannot be expanded.", propertyName));
                }
            }
        }
    }
}

Considerations and Open questions

There are several open questions that have to be discussed and concluded.

Support for other formatters

Should we support $select with json and xml formatters? $expand is not important as these formatters expand by default.
Should we change the behavior of these formatters to not expand by default if the incoming query contains a $expand clause?

Query optimizations

1. Should $select be optimized?

$select is a shape changing query. From a web API perspective, this would mean that the ObjectContent<T> has to be reconstructed. What would happen to the content headers on the ObjectContent? Should we just do a content-negotiation again and say that any content-type headers set after the action finished and before the QueryableAttribute runs would be lost?
An un-optimized $select would mean more memory allocation on the database.

design meeting@Mar 28^th: First cut, actions returning IQueryable will support $select and $expand as it won't be a shape changing query anymore. $select will be optimized.

2. $expand has to be optimized.

Without an optimized $expand no-one would ever use our $expand support unless they are building on top of in-memory IQueryable.
How would the generated query look like? Should we just support EF or try be compatible with some other providers? (Scoping question).
If we do it only for EF, what is the extensibility story for custom providers?

design meeting@Mar 28^th: work item: update with the queries

PageSize for expanded feeds.

How can one specify page size for expanded feeds? WCF DS has a per entity-set configuration knob for page size. Should we follow their model?

Implementation details

public class SelectExpandWrapper<TElement> 
{
    public PropertyContainer Container { get; set; }
}
internal abstract class PropertyContainer
{
        public Dictionary<string, object> ToDictionary(bool includeAutoSelected = true)
}

Also, we wanted to free the OData formatter from having to look at the $select and $expand options. To support that, the wrapper class should contain information about three things,

what structural properties/navigation properties have to be serialized.
what actions have to be included in the response.
what navigation properties have to be expanded.

To support that the wrapper should also implement the IEdmStructuredObject interface.

public interface IEdmStructuredObject : IEdmObject
{
        bool TryGetValue(string propertyName, out object value);
}

public interface IEdmObject
{
        IEdmTypeReference GetEdmType();
}

Generated queries

There are two things to consider here.

OData formatter (in the default case), requires the key properties to generate navigation links. So, even if the client did not ask for key properties explicitly in the request, they have to be fetched from the database. We should probably have an extension point here, so that people who customize link generation can fetch extra properties if they want them for link generation.
OData formatter requires the type name of the entity while writing responses. If we are getting a partial object from the database (wrapper class), we have to include the type name in the wrapper class.

using Table per hierarchy strategy (TPH) which is code first default - tph_dump.txt
using table per type strategy (TPT) - tpt_dump.txt

Also, if you are interested in playing around with the sample, I have the cs file here.

simple $select

uri = "~/Customers?$select=ID";
query = customers.Select<Customer, Wrapper>(c => new Wrapper<int>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    Name1 = "ID",
    Value1 = c.ID
});

Response:

{
  "__type": "NS.Customer",
  "ID": 1
}
{
  "__type": "NS.SpecialCustomer",
  "ID": 2
}

simple $expand

uri = "~/Customers?$expand=Orders";
query = customers.Select(c => new SelectExpandWrapper<Customer>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    Instance = c,
    PropertyContainer = new NamedProperty<Order>
    {
        Name = "Orders",
        Value = c.Orders.Select(o => new SelectExpandWrapper
        {
            TypeName = o is SpecialOrder ? typeNameSpecialOrder : typeNameOrder,
            Instance = o
        })
    }
});

Response:

{
  "__type": "NS.Customer",
  "ID": 1,
  "Name": "Raghu",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 1,
      "Amount": 400
    }
  ]
}
{
  "__type": "NS.SpecialCustomer",
  "ID": 2,
  "Name": "Ram",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 2,
      "Amount": 1000
    }
  ]
}

simple $select and $expand

uri = "~/Customers?$select=Name,Orders&$expand=Orders";
query = customers.Select(c => new SelectExpandWrapper<Customer>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    PropertyContainer = new NamedProperty<string> 
    {
        Name = "Name",
        Value = c.Name,
        Next = new NamedProperty<IEnumerable<Order>>
        {
            Name = "Orders",
            Value = c.Orders.Select<Order, Wrapper>(o => new Wrapper
            {
                TypeName = o is SpecialOrder ? typeNameSpecialOrder : typeNameOrder
                Instance = o
            }
        })
    }
});

Response:
{
  "__type": "NS.Customer",
  "Name": "Raghu",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 1,
      "Amount": 400
    }
  ]
}
{
  "__type": "NS.SpecialCustomer",
  "Name": "Ram",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 2,
      "Amount": 1000
    }
  ]
}

selecting properties from expanded results

uri = "~/Customers?$select=Name,Orders/ID,Orders/Amount&$expand=Orders";
query = customers.Select(c => new SelectExpandWrapper<Customer>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    PropertyContainer = new NamedProperty<string>
    {
        Name = "Name",
        Value = c.Name,
        Next = new NamedProperty<IEnumerable<SelectExpandWrapper<Order>>>
        {
            Name = "Orders",
            Value = c.Orders(o => new SelectExpandWrapper<Order>
            {
                TypeName = o is SpecialOrder ? typeNameSpecialOrder : typeNameOrder,
                PropertyContainer = new NamedProperty<int>
                {
                    Name = "ID",
                    Value = o.ID,
                    Next = new NamedProperty<int> 
                    {
                        Name = "Amount",
                        Value = o.Amount
                    }
                }
            }
    })
});

Response:

{
  "__type": "NS.Customer",
  "Name": "Raghu",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 1,
      "Amount": 400
    }
  ]
}
{
  "__type": "NS.SpecialCustomer",
  "Name": "Ram",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 2,
      "Amount": 1000
    }
  ]
}

expanding properties on derived types

uri = "~/Customers?$expand=NS.SpecialCustomer/SpecialOrders";
query = customers.Select(c => new SelectExpandWrapper<Customer>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    Instance = c,
    PropertyContainer = new NamedProperty<SpecialCustomer>
    {
        Name = c is SpecialCustomer ? "SpecialOrders" : null,
        Value = (c as SpecialCustomer).Orders.Select(o => new SelectExpandWrapper<Order>
        {
            TypeName = o is SpecialOrder ? typeNameSpecialOrder : typeNameOrder,
            Instance = o
        }) //requires null propagation. also don't confuse null and non-existence
    }
});

Response:

{
  "__type": "NS.Customer",
  "ID": 1,
  "Name": "Raghu"
}
{
  "__type": "NS.SpecialCustomer",
  "ID": 2,
  "Name": "Ram",
  "SpecialOrders": [
    {
      "__type": "NS.Order",
      "ID": 2,
      "Amount": 1000
    }
  ]
}

selecting properties on derived types

uri = "~/Customers?$select=Name,NS.SpecialCustomer/SpecialCustomerProperty";
query = customers.Select(c => new SelectExpandWrapper<Customer>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    PropertyContainer = new NamedProperty<string>
    {
        Name = "Name",
        Value = c.Name,
        Next = new PropertyContainer<SpecialCustomer>
        {
            Name = c is SpecialCustomer ? "SpecialCustomerProperty" : null,
            Value = (c as SpecialCustomer).SpecialCustomerProperty // requires null propagation.
        }
    }
});

Response:

{
  "__type": "NS.Customer",
  "Name": "Raghu"
}
{
  "__type": "NS.SpecialCustomer",
  "Name": "Ram",
  "SpecialCustomerProperty": 42
}

expanding properties on expanded derived types

uri = "~/Customers?$select=Name&$expand=Orders,Orders/NS.SpecialOrder/SpecialCustomer";
query = customers.Select<Customer, Wrapper>(c => new SelectExpandWrapper<Customer>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    PropertyContainer = new NamedProperty<string>
    {
        Name = "Name",
        Value = c.Name,
        Next = new NamedProperty<IEnumerable<SelectExpandWrapper<Order>>>
        {
            Name = "Orders",
            Value = c.Orders.Select(o => new SelectExpandWrapper<Order>
            {
                Instance = o,
                Name = o is SpecialOrder ? "SpecialCustomer" : null, // optional property
                Value = new Wrapper
                {
                    Instance = (o as SpecialOrder).SpecialCustomer
                }
            }
        }
    })
});

Response:

{
  "__type": "NS.Customer",
  "Name": "Raghu",
  "Orders": [
    {
      "__type": null,
      "ID": 1,
      "Amount": 400
    }
  ]
}
{
  "__type": "NS.SpecialCustomer",
  "Name": "Ram",
  "Orders": [
    {
      "__type": null,
      "ID": 2,
      "Amount": 1000,
      "SpecialCustomer": {
        "__type": "NS.SpecialCustomer",
        "ID": 2,
        "Name": "Ram",
        "SpecialCustomerProperty": 42
      }
    }
  ]
}

Updated Wiki: $select and $expand support

raghuramn — Thu, 09 May 2013 20:25:19 GMT

$select and $expand support

public class Customer
{
    public int ID { get; set; }

    public string Name { get; set; }

    public virtual ICollection<Order> Orders { get; set; }
}

public class SpecialCustomer : Customer
{
    public int SpecialCustomerProperty { get; set; }

    public virtual ICollection<SpecialOrder> SpecialOrders { get; set; }
}

public class Order
{
    public int ID { get; set; }

    public int Amount { get; set; }

    public Customer Customer { get; set; }
}

public class SpecialOrder : Order
{
    public int SpecialOrderProperty { get; set; }

    public virtual SpecialCustomer SpecialCustomer { get; set; }
}

private static IEdmModel GetModel()
{
    ODataConventionModelBuilder builder = new ODataConventionModelBuilder();
    builder.EntitySet<Customer>("Customers");
    builder.EntitySet<Order>("Orders");
    return builder.GetEdmModel();
}

For example, assuming that a service exposes two entity sets ‘Customers’ and ‘Orders’ and a customer has 1..* (one-to-many) relationship with order, the following URL’s are valid,

Supported scenarios

Not supported scenarios

http://server/Customers(42)/Address?$select=City
Selections of complex types is not supported. (OData V4 might have support for it though)

Considerations and Open questions

There are several open questions that have to be discussed and concluded.

Support for other formatters

Should we support $select with json and xml formatters? $expand is not important as these formatters expand by default.
Should we change the behavior of these formatters to not expand by default if the incoming query contains a $expand clause?

Query optimizations

1. Should $select be optimized?

$select is a shape changing query. From a web API perspective, this would mean that the ObjectContent<T> has to be reconstructed. What would happen to the content headers on the ObjectContent? Should we just do a content-negotiation again and say that any content-type headers set after the action finished and before the QueryableAttribute runs would be lost?
An un-optimized $select would mean more memory allocation on the database.

design meeting@Mar 28^th: First cut, actions returning IQueryable will support $select and $expand as it won't be a shape changing query anymore. $select will be optimized.

2. $expand has to be optimized.

Without an optimized $expand no-one would ever use our $expand support unless they are building on top of in-memory IQueryable.
How would the generated query look like? Should we just support EF or try be compatible with some other providers? (Scoping question).
If we do it only for EF, what is the extensibility story for custom providers?

design meeting@Mar 28^th: work item: update with the queries

PageSize for expanded feeds.

How can one specify page size for expanded feeds? WCF DS has a per entity-set configuration knob for page size. Should we follow their model?

Implementation details

public class SelectExpandWrapper<TElement> 
{
    public PropertyContainer Container { get; set; }
}
internal abstract class PropertyContainer
{
        public Dictionary<string, object> ToDictionary(bool includeAutoSelected = true)
}

Also, we wanted to free the OData formatter from having to look at the $select and $expand options. To support that, the wrapper class should contain information about three things,

what structural properties/navigation properties have to be serialized.
what actions have to be included in the response.
what navigation properties have to be expanded.

To support that the wrapper should also implement the IEdmStructuredObject interface.

public interface IEdmStructuredObject : IEdmObject
{
        bool TryGetValue(string propertyName, out object value);
}

public interface IEdmObject
{
        IEdmTypeReference GetEdmType();
}

Generated queries

There are two things to consider here.

OData formatter (in the default case), requires the key properties to generate navigation links. So, even if the client did not ask for key properties explicitly in the request, they have to be fetched from the database. We should probably have an extension point here, so that people who customize link generation can fetch extra properties if they want them for link generation.
OData formatter requires the type name of the entity while writing responses. If we are getting a partial object from the database (wrapper class), we have to include the type name in the wrapper class.

using Table per hierarchy strategy (TPH) which is code first default - tph_dump.txt
using table per type strategy (TPT) - tpt_dump.txt

Also, if you are interested in playing around with the sample, I have the cs file here.

simple $select

uri = "~/Customers?$select=ID";
query = customers.Select<Customer, Wrapper>(c => new Wrapper<int>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    Name1 = "ID",
    Value1 = c.ID
});

Response:

{
  "__type": "NS.Customer",
  "ID": 1
}
{
  "__type": "NS.SpecialCustomer",
  "ID": 2
}

simple $expand

uri = "~/Customers?$expand=Orders";
query = customers.Select(c => new SelectExpandWrapper<Customer>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    Instance = c,
    PropertyContainer = new NamedProperty<Order>
    {
        Name = "Orders",
        Value = c.Orders.Select(o => new SelectExpandWrapper
        {
            TypeName = o is SpecialOrder ? typeNameSpecialOrder : typeNameOrder,
            Instance = o
        })
    }
});

Response:

{
  "__type": "NS.Customer",
  "ID": 1,
  "Name": "Raghu",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 1,
      "Amount": 400
    }
  ]
}
{
  "__type": "NS.SpecialCustomer",
  "ID": 2,
  "Name": "Ram",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 2,
      "Amount": 1000
    }
  ]
}

simple $select and $expand

uri = "~/Customers?$select=Name,Orders&$expand=Orders";
query = customers.Select(c => new SelectExpandWrapper<Customer>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    PropertyContainer = new NamedProperty<string> 
    {
        Name = "Name",
        Value = c.Name,
        Next = new NamedProperty<IEnumerable<Order>>
        {
            Name = "Orders",
            Value = c.Orders.Select<Order, Wrapper>(o => new Wrapper
            {
                TypeName = o is SpecialOrder ? typeNameSpecialOrder : typeNameOrder
                Instance = o
            }
        })
    }
});

Response:
{
  "__type": "NS.Customer",
  "Name": "Raghu",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 1,
      "Amount": 400
    }
  ]
}
{
  "__type": "NS.SpecialCustomer",
  "Name": "Ram",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 2,
      "Amount": 1000
    }
  ]
}

selecting properties from expanded results

uri = "~/Customers?$select=Name,Orders/ID,Orders/Amount&$expand=Orders";
query = customers.Select(c => new SelectExpandWrapper<Customer>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    PropertyContainer = new NamedProperty<string>
    {
        Name = "Name",
        Value = c.Name,
        Next = new NamedProperty<IEnumerable<SelectExpandWrapper<Order>>>
        {
            Name = "Orders",
            Value = c.Orders(o => new SelectExpandWrapper<Order>
            {
                TypeName = o is SpecialOrder ? typeNameSpecialOrder : typeNameOrder,
                PropertyContainer = new NamedProperty<int>
                {
                    Name = "ID",
                    Value = o.ID,
                    Next = new NamedProperty<int> 
                    {
                        Name = "Amount",
                        Value = o.Amount
                    }
                }
            }
    })
});

Response:

{
  "__type": "NS.Customer",
  "Name": "Raghu",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 1,
      "Amount": 400
    }
  ]
}
{
  "__type": "NS.SpecialCustomer",
  "Name": "Ram",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 2,
      "Amount": 1000
    }
  ]
}

expanding properties on derived types

uri = "~/Customers?$expand=NS.SpecialCustomer/SpecialOrders";
query = customers.Select(c => new SelectExpandWrapper<Customer>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    Instance = c,
    PropertyContainer = new NamedProperty<SpecialCustomer>
    {
        Name = c is SpecialCustomer ? "SpecialOrders" : null,
        Value = (c as SpecialCustomer).Orders.Select(o => new SelectExpandWrapper<Order>
        {
            TypeName = o is SpecialOrder ? typeNameSpecialOrder : typeNameOrder,
            Instance = o
        }) //requires null propagation. also don't confuse null and non-existence
    }
});

Response:

{
  "__type": "NS.Customer",
  "ID": 1,
  "Name": "Raghu"
}
{
  "__type": "NS.SpecialCustomer",
  "ID": 2,
  "Name": "Ram",
  "SpecialOrders": [
    {
      "__type": "NS.Order",
      "ID": 2,
      "Amount": 1000
    }
  ]
}

selecting properties on derived types

uri = "~/Customers?$select=Name,NS.SpecialCustomer/SpecialCustomerProperty";
query = customers.Select(c => new SelectExpandWrapper<Customer>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    PropertyContainer = new NamedProperty<string>
    {
        Name = "Name",
        Value = c.Name,
        Next = new PropertyContainer<SpecialCustomer>
        {
            Name = c is SpecialCustomer ? "SpecialCustomerProperty" : null,
            Value = (c as SpecialCustomer).SpecialCustomerProperty // requires null propagation.
        }
    }
});

Response:

{
  "__type": "NS.Customer",
  "Name": "Raghu"
}
{
  "__type": "NS.SpecialCustomer",
  "Name": "Ram",
  "SpecialCustomerProperty": 42
}

expanding properties on expanded derived types

uri = "~/Customers?$select=Name&$expand=Orders,Orders/NS.SpecialOrder/SpecialCustomer";
query = customers.Select<Customer, Wrapper>(c => new SelectExpandWrapper<Customer>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    PropertyContainer = new NamedProperty<string>
    {
        Name = "Name",
        Value = c.Name,
        Next = new NamedProperty<IEnumerable<SelectExpandWrapper<Order>>>
        {
            Name = "Orders",
            Value = c.Orders.Select(o => new SelectExpandWrapper<Order>
            {
                Instance = o,
                Name = o is SpecialOrder ? "SpecialCustomer" : null, // optional property
                Value = new Wrapper
                {
                    Instance = (o as SpecialOrder).SpecialCustomer
                }
            }
        }
    })
});

Response:

{
  "__type": "NS.Customer",
  "Name": "Raghu",
  "Orders": [
    {
      "__type": null,
      "ID": 1,
      "Amount": 400
    }
  ]
}
{
  "__type": "NS.SpecialCustomer",
  "Name": "Ram",
  "Orders": [
    {
      "__type": null,
      "ID": 2,
      "Amount": 1000,
      "SpecialCustomer": {
        "__type": "NS.SpecialCustomer",
        "ID": 2,
        "Name": "Ram",
        "SpecialCustomerProperty": 42
      }
    }
  ]
}

Updated Wiki: $select and $expand support

raghuramn — Thu, 09 May 2013 20:18:23 GMT

$select and $expand support

public class Customer
{
    public int ID { get; set; }

    public string Name { get; set; }

    public virtual ICollection<Order> Orders { get; set; }
}

public class SpecialCustomer : Customer
{
    public int SpecialCustomerProperty { get; set; }

    public virtual ICollection<SpecialOrder> SpecialOrders { get; set; }
}

public class Order
{
    public int ID { get; set; }

    public int Amount { get; set; }

    public Customer Customer { get; set; }
}

public class SpecialOrder : Order
{
    public int SpecialOrderProperty { get; set; }

    public virtual SpecialCustomer SpecialCustomer { get; set; }
}

private static IEdmModel GetModel()
{
    ODataConventionModelBuilder builder = new ODataConventionModelBuilder();
    builder.EntitySet<Customer>("Customers");
    builder.EntitySet<Order>("Orders");
    return builder.GetEdmModel();
}

For example, assuming that a service exposes two entity sets ‘Customers’ and ‘Orders’ and a customer has 1..* (one-to-many) relationship with order, the following URL’s are valid,

Supported scenarios

Not supported scenarios

http://server/Customers(42)/Address?$select=City
Selections of complex types is not supported. (OData V4 might have support for it though)

Considerations and Open questions

There are several open questions that have to be discussed and concluded.

Support for other formatters

Should we support $select with json and xml formatters? $expand is not important as these formatters expand by default.
Should we change the behavior of these formatters to not expand by default if the incoming query contains a $expand clause?

Query optimizations

1. Should $select be optimized?

$select is a shape changing query. From a web API perspective, this would mean that the ObjectContent<T> has to be reconstructed. What would happen to the content headers on the ObjectContent? Should we just do a content-negotiation again and say that any content-type headers set after the action finished and before the QueryableAttribute runs would be lost?
An un-optimized $select would mean more memory allocation on the database.

design meeting@Mar 28^th: First cut, actions returning IQueryable will support $select and $expand as it won't be a shape changing query anymore. $select will be optimized.

2. $expand has to be optimized.

Without an optimized $expand no-one would ever use our $expand support unless they are building on top of in-memory IQueryable.
How would the generated query look like? Should we just support EF or try be compatible with some other providers? (Scoping question).
If we do it only for EF, what is the extensibility story for custom providers?

design meeting@Mar 28^th: work item: update with the queries

PageSize for expanded feeds.

How can one specify page size for expanded feeds? WCF DS has a per entity-set configuration knob for page size. Should we follow their model?

Implementation details

public class SelectExpandWrapper<TElement> 
{
    public PropertyContainer Container { get; set; }
}
internal abstract class PropertyContainer
{
        public Dictionary<string, object> ToDictionary(bool includeAutoSelected = true)
}

Also, we wanted to free the OData formatter from having to look at the $select and $expand options. To support that, the wrapper class should contain information about three things,

what structural properties/navigation properties have to be serialized.
what actions have to be included in the response.
what navigation properties have to be expanded.

To support that the wrapper should also implement the IEdmStructuredObject interface.

public interface IEdmStructuredObject : IEdmObject
{
        bool TryGetValue(string propertyName, out object value);
}

public interface IEdmObject
{
        IEdmTypeReference GetEdmType();
}

Generated queries

There are two things to consider here.

OData formatter (in the default case), requires the key properties to generate navigation links. So, even if the client did not ask for key properties explicitly in the request, they have to be fetched from the database. We should probably have an extension point here, so that people who customize link generation can fetch extra properties if they want them for link generation.
OData formatter requires the type name of the entity while writing responses. If we are getting a partial object from the database (wrapper class), we have to include the type name in the wrapper class.

using Table per hierarchy strategy (TPH) which is code first default - tph_dump.txt
using table per type strategy (TPT) - tpt_dump.txt

Also, if you are interested in playing around with the sample, I have the cs file here.

simple $select

uri = "~/Customers?$select=ID";
query = customers.Select<Customer, Wrapper>(c => new Wrapper<int>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    Name1 = "ID",
    Value1 = c.ID
});

Response:

{
  "__type": "NS.Customer",
  "ID": 1
}
{
  "__type": "NS.SpecialCustomer",
  "ID": 2
}

simple $expand

uri = "~/Customers?$expand=Orders";
query = customers.Select(c => new SelectExpandWrapper<Customer>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    Instance = c,
    PropertyContainer = new NamedProperty<Order>
    {
        Name = "Orders",
        Value = c.Orders.Select(o => new SelectExpandWrapper
        {
            TypeName = o is SpecialOrder ? typeNameSpecialOrder : typeNameOrder,
            Instance = o
        })
    }
});

Response:

{
  "__type": "NS.Customer",
  "ID": 1,
  "Name": "Raghu",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 1,
      "Amount": 400
    }
  ]
}
{
  "__type": "NS.SpecialCustomer",
  "ID": 2,
  "Name": "Ram",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 2,
      "Amount": 1000
    }
  ]
}

simple $select and $expand

uri = "~/Customers?$select=Name,Orders&$expand=Orders";
query = customers.Select(c => new SelectExpandWrapper<Customer>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    PropertyContainer = new NamedProperty<string> 
    {
        Name = "Name",
        Value = c.Name,
        Next = new NamedProperty<IEnumerable<Order>>
        {
            Name = "Orders",
            Value = c.Orders.Select<Order, Wrapper>(o => new Wrapper
            {
                TypeName = o is SpecialOrder ? typeNameSpecialOrder : typeNameOrder
                Instance = o
            }
        })
    }
});

Response:
{
  "__type": "NS.Customer",
  "Name": "Raghu",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 1,
      "Amount": 400
    }
  ]
}
{
  "__type": "NS.SpecialCustomer",
  "Name": "Ram",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 2,
      "Amount": 1000
    }
  ]
}

selecting properties from expanded results

uri = "~/Customers?$select=Name,Orders/ID,Orders/Amount&$expand=Orders";
query = customers.Select(c => new SelectExpandWrapper
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    PropertyContainer = new NamedProperty<string>
    {
        Name = "Name",
        Value = c.Name,
        Next = new NamedProperty<IEnumerable<SelectExpandWrapper<Order>>>
        {
            Name = "Orders",
            Value = c.Orders(o => new SelectExpandWrapper<Order>
            {
                TypeName = o is SpecialOrder ? typeNameSpecialOrder : typeNameOrder,
                PropertyContainer = new NamedProperty<int>
                {
                    Name = "ID",
                    Value = o.ID,
                    Next = new NamedProperty<int> 
                    {
                        Name = "Amount",
                        Value = o.Amount
                    }
                }
            }
    })
});

Response:

{
  "__type": "NS.Customer",
  "Name": "Raghu",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 1,
      "Amount": 400
    }
  ]
}
{
  "__type": "NS.SpecialCustomer",
  "Name": "Ram",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 2,
      "Amount": 1000
    }
  ]
}

expanding properties on derived types

uri = "~/Customers?$expand=NS.SpecialCustomer/SpecialOrders";
query = customers.Select<Customer, Wrapper>(c => new Wrapper<IEnumerable<Wrapper>>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    AllStrucuturalProperties = "ID,Name",
    Instance = c,
    Name1 = c is SpecialCustomer ? "SpecialOrders" : null,
    Value1 = (c as SpecialCustomer).Orders.Select<Order, Wrapper>(o => new Wrapper
    {
        TypeName = o is SpecialOrder ? typeNameSpecialOrder : typeNameOrder,
        AllStrucuturalProperties = "ID,Amount",
        Instance = o
    }) //requires null propagation. also don't confuse null and non-existence
});

Response:

{
  "__type": "NS.Customer",
  "ID": 1,
  "Name": "Raghu"
}
{
  "__type": "NS.SpecialCustomer",
  "ID": 2,
  "Name": "Ram",
  "SpecialOrders": [
    {
      "__type": "NS.Order",
      "ID": 2,
      "Amount": 1000
    }
  ]
}

selecting properties on derived types

uri = "~/Customers?$select=Name,NS.SpecialCustomer/SpecialCustomerProperty";
query = customers.Select<Customer, Wrapper>(c => new Wrapper<string, int?>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    Name1 = "Name",
    Value1 = c.Name,
    // requires null propagation. also don't confuse null and non-existence. If the property name is null, property is non 
    Name2 = c is SpecialCustomer ? "SpecialCustomerProperty" : null,
    Value2 = (c as SpecialCustomer).SpecialCustomerProperty // notice the int? - required because materializer fails without it
});

Response:

{
  "__type": "NS.Customer",
  "Name": "Raghu"
}
{
  "__type": "NS.SpecialCustomer",
  "Name": "Ram",
  "SpecialCustomerProperty": 42
}

expanding properties on expanded derived types

uri = "~/Customers?$select=Name&$expand=Orders,Orders/NS.SpecialOrder/SpecialCustomer";
query = customers.Select<Customer, Wrapper>(c => new Wrapper<string, IEnumerable<Wrapper>>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    Name1 = "Name",
    Value1 = c.Name,
    Name2 = "Orders",
    Value2 = c.Orders.Select<Order, Wrapper>(o => new Wrapper<Wrapper>
    {
        AllStrucuturalProperties = "ID,Amount",
        Instance = o,
        Name1 = o is SpecialOrder ? "SpecialCustomer" : null, // optional property
        Value1 = new Wrapper
        {
            TypeName = (o as SpecialOrder).SpecialCustomer is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
            AllStrucuturalProperties = "ID,Name,SpecialCustomerProperty",
            Instance = (o as SpecialOrder).SpecialCustomer
        }
    })
});

Response:

{
  "__type": "NS.Customer",
  "Name": "Raghu",
  "Orders": [
    {
      "__type": null,
      "ID": 1,
      "Amount": 400
    }
  ]
}
{
  "__type": "NS.SpecialCustomer",
  "Name": "Ram",
  "Orders": [
    {
      "__type": null,
      "ID": 2,
      "Amount": 1000,
      "SpecialCustomer": {
        "__type": "NS.SpecialCustomer",
        "ID": 2,
        "Name": "Ram",
        "SpecialCustomerProperty": 42
      }
    }
  ]
}

Updated Wiki: $select and $expand support

raghuramn — Thu, 09 May 2013 16:30:42 GMT

$select and $expand support

public class Customer
{
    public int ID { get; set; }

    public string Name { get; set; }

    public virtual ICollection<Order> Orders { get; set; }
}

public class SpecialCustomer : Customer
{
    public int SpecialCustomerProperty { get; set; }

    public virtual ICollection<SpecialOrder> SpecialOrders { get; set; }
}

public class Order
{
    public int ID { get; set; }

    public int Amount { get; set; }

    public Customer Customer { get; set; }
}

public class SpecialOrder : Order
{
    public int SpecialOrderProperty { get; set; }

    public virtual SpecialCustomer SpecialCustomer { get; set; }
}

private static IEdmModel GetModel()
{
    ODataConventionModelBuilder builder = new ODataConventionModelBuilder();
    builder.EntitySet<Customer>("Customers");
    builder.EntitySet<Order>("Orders");
    return builder.GetEdmModel();
}

For example, assuming that a service exposes two entity sets ‘Customers’ and ‘Orders’ and a customer has 1..* (one-to-many) relationship with order, the following URL’s are valid,

Supported scenarios

Not supported scenarios

http://server/Customers(42)/Address?$select=City
Selections of complex types is not supported. (OData V4 might have support for it though)

Considerations and Open questions

There are several open questions that have to be discussed and concluded.

Support for other formatters

Should we support $select with json and xml formatters? $expand is not important as these formatters expand by default.
Should we change the behavior of these formatters to not expand by default if the incoming query contains a $expand clause?

Query optimizations

1. Should $select be optimized?

$select is a shape changing query. From a web API perspective, this would mean that the ObjectContent<T> has to be reconstructed. What would happen to the content headers on the ObjectContent? Should we just do a content-negotiation again and say that any content-type headers set after the action finished and before the QueryableAttribute runs would be lost?
An un-optimized $select would mean more memory allocation on the database.

design meeting@Mar 28^th: First cut, actions returning IQueryable will support $select and $expand as it won't be a shape changing query anymore. $select will be optimized.

2. $expand has to be optimized.

Without an optimized $expand no-one would ever use our $expand support unless they are building on top of in-memory IQueryable.
How would the generated query look like? Should we just support EF or try be compatible with some other providers? (Scoping question).
If we do it only for EF, what is the extensibility story for custom providers?

design meeting@Mar 28^th: work item: update with the queries

PageSize for expanded feeds.

How can one specify page size for expanded feeds? WCF DS has a per entity-set configuration knob for page size. Should we follow their model?

Implementation details

public class SelectExpandWrapper<TElement> 
{
    public PropertyContainer Container { get; set; }
}
internal abstract class PropertyContainer
{
        public Dictionary<string, object> ToDictionary(bool includeAutoSelected = true)
}

Also, we wanted to free the OData formatter from having to look at the $select and $expand options. To support that, the wrapper class should contain information about three things,

what structural properties/navigation properties have to be serialized.
what actions have to be included in the response.
what navigation properties have to be expanded.

To support that the wrapper should also implement the IEdmStructuredObject interface.

public interface IEdmStructuredObject : IEdmObject
{
        bool TryGetValue(string propertyName, out object value);
}

public interface IEdmObject
{
        IEdmTypeReference GetEdmType();
}

Generated queries

There are two things to consider here.

OData formatter (in the default case), requires the key properties to generate navigation links. So, even if the client did not ask for key properties explicitly in the request, they have to be fetched from the database. We should probably have an extension point here, so that people who customize link generation can fetch extra properties if they want them for link generation.
OData formatter requires the type name of the entity while writing responses. If we are getting a partial object from the database (wrapper class), we have to include the type name in the wrapper class.

using Table per hierarchy strategy (TPH) which is code first default - tph_dump.txt
using table per type strategy (TPT) - tpt_dump.txt

Also, if you are interested in playing around with the sample, I have the cs file here.

simple $select

uri = "~/Customers?$select=ID";
query = customers.Select<Customer, Wrapper>(c => new Wrapper<int>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    Name1 = "ID",
    Value1 = c.ID
});

Response:

{
  "__type": "NS.Customer",
  "ID": 1
}
{
  "__type": "NS.SpecialCustomer",
  "ID": 2
}

simple $expand

uri = "~/Customers?$expand=Orders";
query = customers.Select<Customer, Wrapper>(c => new Wrapper<IEnumerable<Wrapper>>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    AllStrucuturalProperties = "ID,Name",
    Instance = c,
    Name1 = "Orders",
    Value1 = c.Orders.Select<Order, Wrapper>(o => new Wrapper
    {
        TypeName = o is SpecialOrder ? typeNameSpecialOrder : typeNameOrder,
        AllStrucuturalProperties = "ID,Amount",
        Instance = o
    })
});

Response:

{
  "__type": "NS.Customer",
  "ID": 1,
  "Name": "Raghu",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 1,
      "Amount": 400
    }
  ]
}
{
  "__type": "NS.SpecialCustomer",
  "ID": 2,
  "Name": "Ram",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 2,
      "Amount": 1000
    }
  ]
}

simple $select and $expand

uri = "~/Customers?$select=Name,Orders&$expand=Orders";
query = customers.Select<Customer, Wrapper>(c => new Wrapper<string, IEnumerable<Wrapper>>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    Name1 = "Name",
    Value1 = c.Name,
    Name2 = "Orders",
    Value2 = c.Orders.Select<Order, Wrapper>(o => new Wrapper
    {
        TypeName = o is SpecialOrder ? typeNameSpecialOrder : typeNameOrder
        AllStrucuturalProperties = "ID,Amount",
        Instance = o
    })
});

Response:
{
  "__type": "NS.Customer",
  "Name": "Raghu",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 1,
      "Amount": 400
    }
  ]
}
{
  "__type": "NS.SpecialCustomer",
  "Name": "Ram",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 2,
      "Amount": 1000
    }
  ]
}

selecting properties from expanded results

uri = "~/Customers?$select=Name,Orders/ID,Orders/Amount&$expand=Orders";
query = customers.Select<Customer, Wrapper>(c => new Wrapper<string, IEnumerable<Wrapper>>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    Name1 = "Name",
    Value1 = c.Name,
    Name2 = "Orders",
    Value2 = c.Orders.Select<Order, Wrapper>(o => new Wrapper<int, int>
    {
        TypeName = o is SpecialOrder ? typeNameSpecialOrder : typeNameOrder,
        Name1 = "ID",
        Name2 = "Amount",
        Value1 = o.ID,
        Value2 = o.Amount
    })
});

Response:

{
  "__type": "NS.Customer",
  "Name": "Raghu",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 1,
      "Amount": 400
    }
  ]
}
{
  "__type": "NS.SpecialCustomer",
  "Name": "Ram",
  "Orders": [
    {
      "__type": "NS.Order",
      "ID": 2,
      "Amount": 1000
    }
  ]
}

expanding properties on derived types

uri = "~/Customers?$expand=NS.SpecialCustomer/SpecialOrders";
query = customers.Select<Customer, Wrapper>(c => new Wrapper<IEnumerable<Wrapper>>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    AllStrucuturalProperties = "ID,Name",
    Instance = c,
    Name1 = c is SpecialCustomer ? "SpecialOrders" : null,
    Value1 = (c as SpecialCustomer).Orders.Select<Order, Wrapper>(o => new Wrapper
    {
        TypeName = o is SpecialOrder ? typeNameSpecialOrder : typeNameOrder,
        AllStrucuturalProperties = "ID,Amount",
        Instance = o
    }) //requires null propagation. also don't confuse null and non-existence
});

Response:

{
  "__type": "NS.Customer",
  "ID": 1,
  "Name": "Raghu"
}
{
  "__type": "NS.SpecialCustomer",
  "ID": 2,
  "Name": "Ram",
  "SpecialOrders": [
    {
      "__type": "NS.Order",
      "ID": 2,
      "Amount": 1000
    }
  ]
}

selecting properties on derived types

uri = "~/Customers?$select=Name,NS.SpecialCustomer/SpecialCustomerProperty";
query = customers.Select<Customer, Wrapper>(c => new Wrapper<string, int?>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    Name1 = "Name",
    Value1 = c.Name,
    // requires null propagation. also don't confuse null and non-existence. If the property name is null, property is non 
    Name2 = c is SpecialCustomer ? "SpecialCustomerProperty" : null,
    Value2 = (c as SpecialCustomer).SpecialCustomerProperty // notice the int? - required because materializer fails without it
});

Response:

{
  "__type": "NS.Customer",
  "Name": "Raghu"
}
{
  "__type": "NS.SpecialCustomer",
  "Name": "Ram",
  "SpecialCustomerProperty": 42
}

expanding properties on expanded derived types

uri = "~/Customers?$select=Name&$expand=Orders,Orders/NS.SpecialOrder/SpecialCustomer";
query = customers.Select<Customer, Wrapper>(c => new Wrapper<string, IEnumerable<Wrapper>>
{
    TypeName = c is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
    Name1 = "Name",
    Value1 = c.Name,
    Name2 = "Orders",
    Value2 = c.Orders.Select<Order, Wrapper>(o => new Wrapper<Wrapper>
    {
        AllStrucuturalProperties = "ID,Amount",
        Instance = o,
        Name1 = o is SpecialOrder ? "SpecialCustomer" : null, // optional property
        Value1 = new Wrapper
        {
            TypeName = (o as SpecialOrder).SpecialCustomer is SpecialCustomer ? typeNameSpecialCustomer : typeNameCustomer,
            AllStrucuturalProperties = "ID,Name,SpecialCustomerProperty",
            Instance = (o as SpecialOrder).SpecialCustomer
        }
    })
});

Response:

{
  "__type": "NS.Customer",
  "Name": "Raghu",
  "Orders": [
    {
      "__type": null,
      "ID": 1,
      "Amount": 400
    }
  ]
}
{
  "__type": "NS.SpecialCustomer",
  "Name": "Ram",
  "Orders": [
    {
      "__type": null,
      "ID": 2,
      "Amount": 1000,
      "SpecialCustomer": {
        "__type": "NS.SpecialCustomer",
        "ID": 2,
        "Name": "Ram",
        "SpecialCustomerProperty": 42
      }
    }
  ]
}

New Comment on "CORS support for ASP.NET Web API"

ramnaresh_t — Thu, 09 May 2013 16:10:13 GMT

Excited when it will be part of WEB API..

New Comment on "Roadmap"

Rujaun — Sun, 05 May 2013 18:26:28 GMT

Thanks for all the hard work! I'm really glad that you guys are trying to simplify the use of DropDownLists, Its a real pain in the ass to make it work, and not allot of helpful tutorials and info on it. And very smart move to integrate twitter bootstrap! This is why I love to be a .net developer =)

New Comment on "Roadmap"

Rujaun — Sun, 05 May 2013 18:25:23 GMT

New Comment on "Attribute routing in Web API"

aliostad — Mon, 29 Apr 2013 07:44:40 GMT

"Give a rope to people, they will hang themselves up with it" - anonymous I have meant to write this comment weeks before but been busy and I hope it is not too late. Just fresh from finding a big performance bottleneck which turned out to be due to AttributeRouting, I think it is time to say a few words. Routing is an area which needs improvement but the direction taken by adopting AttributeRouting is the opposite of what is needed. So instead of decentralising routes and hide the valuable information in string tokens, we need to move towards not only centralising the routing (as it is a strategic application-wide decision) but to allow the routes to be defined with zero configuration by the resources. And also the relationships to be defined naturally by the resources. So I believe it is definitely more like Darrel Miller's ApiRouter rather than AttributeRouting. Another aspect is the hierarchical routing which also defines the relationships naturally with code rather than arbitrary string tokens. This allows hypermedia to be derived from the resource itself rather than to hand-coded in the resource. Reality is Web APIs are hierarchical while routing adopted from MVC is very flat. One of the problems is the linear search through the routes definitely does not scale. I can do a proof of concept to show what I mean - if you think it is useful.

New Comment on "Attribute routing in Web API"

aliostad — Mon, 29 Apr 2013 07:39:24 GMT

New Comment on "CORS support for ASP.NET Web API"

Geminiman — Fri, 26 Apr 2013 21:22:06 GMT

The latest nightlies has parameters on the EnableCorsAttribute. What should I be putting in for origin, headers and methods to make all of them work? (I just want to allow a request from anyone with any method type and any headers.)

New Comment on "Attribute routing in Web API"

danroth27 — Fri, 26 Apr 2013 17:32:08 GMT

The good news is that Tim McCall of AttributeRouting fame is the main contributor to pulling attribute into ASP.NET Web API, which means the design benefits from his extensive expertise. There is no intent to reinvent the API. Instead the goal is to integrate the API right into the runtime so that attribute routing becomes a first class experience (ex. we are integrating directly with the existing HttpGet/Post/Put/Delete attributes instead of creating new ones). That said, your feedback is critical to ensuring we do our job right, so by all means please grab a daily build and let us know what you think!

New Comment on "Attribute routing in Web API"

aliostad — Fri, 26 Apr 2013 15:59:41 GMT

"Give a rope to people, they will hang themselves up with it" - anonymous I meant to write this comment weeks before but been busy and I hope it is not too late. Just fresh from finding a big performance bottleneck which turned out to be due to AttributeRouting, I think it is time to say a few words. Routing is an area which needs improvement but the direction taken by adopting AttributeRouting is the opposite of what is needed. So instead of decentralising routes and hide the valuable information in string tokens, we need to move towards not only centralising the routing (as it is a strategic application-wide decision) but to allow the routes to be defined with zero configuration by the resources. And also the relationships to be defined naturally by the resources. So I believe it is definitely more like Darrel Miller's ApiRouter rather than AttributeRouting. Another aspect is the hierarchical routing which also defines the relationships naturally with code rather than arbitrary string tokens. This allows hypermedia to be derived from the resource itself rather than to hand-coded in the resource. Reality is Web APIs are hierarchical while routing adopted from MVC is very flat. One of the problems is the linear search through the routes definitely does not scale. I can do a proof of concept to show what I mean - if you think it is useful.

Updated Wiki: Contributors

yaohuang — Thu, 25 Apr 2013 23:11:49 GMT

We would like to thank the following contributors for sharing code and making ASP.NET MVC, Web API and Web Pages better products.

New Comment on "Attribute routing in Web API"

corinblaikie — Wed, 24 Apr 2013 11:16:44 GMT

It would be nice if the routes could also be defined on interfaces too. The use case for this is having an interface that defines the API and client classes / server classes that implement the interface.

New Comment on "CORS support for ASP.NET Web API"

mocsharp — Sat, 20 Apr 2013 18:29:19 GMT

Is this available for self-hosted?