Make your EF Query logs traceable

If you are a .NET developer, there is high chance that you’ve spent way too much time looking at Entity Framework logs to understand and investigate generated queries. Probably one of the most time-consuming tasks is to find out what has been translated to what especially when it comes to complex chatty solutions. Entity Framework Core 2.2 introduced a new feature called Query Tags which can be used to tag queries and make the output traceable. You simply can annotate your LINQ queries using TagWith() method.

Let’s have a look at an example. In the following example, I annotated my query with “List of top 10 books”:

var books =
      (from b in DbContext.Books.TagWith(@"List of top 10 books")
      orderby b.Price descending
      select b).Take(10).ToList();

And here is the query output in the the log:

-- List of top 10 books
 
SELECT TOP(@__p_1) [b].[Id], [b].[Name], [b].[Price]
FROM [Books] AS [b]
ORDER BY [b].[Price] DESC

Now as a developer, I can easily find out my query output in logs without burning calories!

Estimating Azure DocumentDB Throughput Needs

Azure_estimating-azure-documentdb-throughput-needs_01

When you want to create a collection in your Azure DocumentDB database account, you have to estimate and specify the throughput size and Azure DocumentDB reserves resources to satisfy your application throughput’s needs. You have to pay for the reserved resources allocated to your collection regardless of usage and the is why it’s important to estimate the throughput size correctly to reduce the operation costs.

What is the throughput measure unit?
Request Unit (RU) per second is the unit of throughput measurement. Azure reserves specified amount of RU/S as your collection throughput. A single request unit represents the processing capacity required to read a single 1KB document. Depends on your document, other requests such as create, update and delete consume more request units.

Specifying throughput size
You have to specify throughput size when you want to create a collection

How to measure throughput size ?
As a first step it’s always good to start with default throughout size. and then start monitoring and measuring consumed request units for common operations and adjust the throughput size. When you query against a collection, Azure returns request charge value in portal or through x-ms-request-charge response header in code. Therefore, you can get some ideas about cost of  your queries.

Azure portal returns request charge
You can see the request charge in Azure portal

Many factors are involved in request unit measurement. Things like number of document properties, indexes, document size and data consistency. Therefore, RU cost differs from application to another application. When you have an idea about your application queries costs and estimated number of requests per second you can estimate how much throughput you need to satisfy your application needs.

Set Alarms
One of the best methods to monitor your service performance and consumed RUs is to set alarms. You can define as many types of alarm as you want to make sure reserved throughput is enough but not more than enough. As a case in point, you can monitor throttled requests number to make sure enough resources has been allocated or check consumed request units to make sure your are not allocating more than enough. If you are keep getting notifications about not consuming expected RUs then it’s time to scale down.

Setting alarms
Administrator is being notified when consumed RUs is less than expected

Brief overview of Azure DocumentDB Document Properties

When you create a document in Azure DocumentDB, regardless of properties in the document, Azure creates and populates some default properties behind the scene for the document. Here is a brief overview of document properties:

_RID: Indicates unique resource identifier. Each document in Azure DocumentDB must have a unique string identifier (across the collection) which can be changed over the time and is specified in the document; However, _RID is generated by Azure and cannot be changed over the time.

_TS: Indicates the last time when document gets modified. Azure updates the value when you modify the document.  This can be very useful for  your application to get latest changes and updates after specific date.

_SELF: Unique addressable URI of the resource. Here is an example of the _SELF link:
dbs/DppTAA==/colls/DppTAvdAA=/docs/DppTAMYvdAABAAAAAAA==/
As you can see it starts with dbs and database Resource ID and then Colls and the collection Resource ID and finally docs and the document _RID identifier.

_ETAG: _ETAGs are used by Azure to manage optimistic concurrency and avoid users override each others’ changes. Stas Kondratyev posted a comprehensive article about _ETAGS here.

_ATTACHMENTS: Indicates path to the document’s attachment.

How to get Document Properties ?

To get the document properties, you need to select the document you want from the collection and then click on Properties icon on Document blade and you can get all the properties in Properties blade.

Cognitive-Services_Computer-Vision-API_04