Retrieving Metadata in D365 / CDS

One of the things that really attracts me to the D365 platform is its metadata capabilities. As an ISV we make a lot of use of this at Data8, making sure our solutions automatically adapt to the customizations that each customer has made in their own system.

There are a few different ways to access the metadata, from both the SDK and Web API. I’m going to look at the SDK options in this post – this is what you’d use in a plugin or external app.

There are three requests in the SDK to access metadata:

• RetrieveEntityRequest
• RetrieveAllEntitiesRequest
• RetrieveMetadataChangesRequest

These are all perfectly valid options, but it’s worth having a closer look to make sure you use them as efficiently as possible.

RetrieveEntityRequest

This gets you the metadata for a single entity, based on its logical name:

var metadata = ((RetrieveEntityResponse)svc.Execute(new RetrieveEntityRequest
{
    LogicalName = "account",
    EntityFilters = EntityFilters.Default
})).EntityMetadata;

The EntityFilters are key here. These select how much information you want to get back in the response:

• Entity or Default (these two mean the same thing). This gives you summary information about the entity itself (display name etc.)
• Attributes gets the details of the individual attributes within the entity
• Relationships includes the details of the 1:N, N:1 and N:N relationships that the entity is involved in
• Privileges gets the details of the privileges that apply to the entity

You can combine these together to get the information you need, e.g. Entity | Attributes. You can also select All as a shorthand for Entity | Attributes | Relationships | Privileges.

Regardless of what options you select, you’ll still get an EntityMetadata object back, but some fields will be populated and some will be left as null.

Of course you need to select at least as much information as your application requires, but don’t fall into the trap of selecting EntityFilters.All because it’s easy – it can really hurt the performance of your system.

You can see that each extra option adds to the response time, but getting the attributes is the biggest change. This example was run using the account entity which has a lot of attributes so this might not be representative of all entities, but it still highlights the important point: do not ask for more information than you need!

RetrieveAllEntitiesRequest

This request works identically to RetrieveEntityRequest, but gives you the requested details for all entities. So if you want to get a listing of all available entities and their attributes you could use:

var metadatas = ((RetrieveAllEntitiesResponse)svc.Execute(new RetrieveAllEntitiesRequest
{
    EntityFilters = EntityFilters.Entity | EntityFilters.Attributes
})).EntityMetadata;

foreach (var metadata in metadatas)
{
    // ...
}

If we have a look at the performance of this method with the same combinations of EntityFilters as before we get a similar shaped chart:

Again we see the biggest jump in elapsed time when we add in the Attributes option. Look at the horizontal access though – this time it costs us almost 15 seconds! That’s a lot of time for any sort of interactive application, so you really need to avoid this wherever possible.

RetrieveMetadataChangesRequest

This is the big one that gives us way more flexibility in what we request compared to the other two. It’s also one I’d previously discounted without investigating too much due to its name. I’d previously considered that it was only useful for getting the changes in metadata over time, which is useful for applications that need to synchronise data and metadata for long periods but not for single use.

It turns out though that, while you can give this method some details to get the changes since the previous time you called it, you can also use it as a powerful standalone request to get a very precisely filtered set of metadata.

Example

The key to using this request is the EntityQueryExpression. If you’re familiar with querying data using a QueryExpression this will seem familiar to you.

var metadatas = ((RetrieveMetadataChangesResponse)svc.Execute(new RetrieveMetadataChangesRequest
{
  Query = new EntityQueryExpression
  {
    Criteria = new MetadataFilterExpression
    {
      Conditions =
      {
        new MetadataConditionExpression(nameof(EntityMetadata.LogicalName), MetadataConditionOperator.Equals, "account")
      }
    },
    Properties = new MetadataPropertiesExpression
    {
      PropertyNames =
      {
        nameof(EntityMetadata.LogicalCollectionName),
        nameof(EntityMetadata.Attributes)
      }
    },
    AttributeQuery = new AttributeQueryExpression
    {
      Criteria = new MetadataFilterExpression
      {
        Conditions =
        {
          new MetadataConditionExpression(nameof(AttributeMetadata.IsValidForRead), MetadataConditionOperator.Equals, true),
          new MetadataConditionExpression(nameof(AttributeMetadata.AttributeType), MetadataConditionOperator.Equals, AttributeTypeCode.Boolean)
        }
      },
      Properties = new MetadataPropertiesExpression
      {
        PropertyNames =
        {
          nameof(AttributeMetadata.LogicalName)
        }
      }
    }
  }
})).EntityMetadata;

This is really two queries in one. In the first part we set the Criteria that we want to use to filter the list of entities, and the list of Properties of those entities that we want to get back. Unlike the earlier options we get to pick and choose exactly which properties to get, rather than having them defined in blocks with the EntityFilters enum.

In the second part we do the same again but this time on the attributes within those entities. In this case we’re only going to get a list of the names of readable boolean attributes in the account entity, and also the collection name for the entity that we’d use in a WebAPI query.

There are also similar queries you can add in to select details of the relationships, keys and labels within the metadata too.

Query Syntax

When you select the names of properties to include or filter by, you can just use a simple string like:

new MetadataConditionExpression("LogicalName", MetadataConditionOperator.Equals, "account")

However, I prefer to use the nameof operator for a few reasons:

• stops typos in your string literals
• get Intellisense on the available properties
• “Find References” will identify this usage

Performance

I’ve done a couple of example queries to compare the performance of this method. In the first one I got the logical collection name of the entity and the details of the name attribute for the account entity. In the second I got a list of all readable boolean properties across all entities:

Again, getting more attributes from more entities takes longer, but I manage to limit the time here to below 2 seconds by being more selective. Compare this to the 15+ seconds it took to get all the attributes from all entities using RetrieveAllEntitiesRequest.

Overall

Putting these all together we can look at the relative performance of each method:

The simple rule is you get what you pay for – the more information you need, the longer it’ll take to get it. But if you need information about attributes from more than one entity, but you don’t need all attributes from all entities, consider using RetrieveMetadataChangesRequest instead of RetrieveAllEntitiesRequest for potentially a massive performance improvement.