How To

Structured JavaScript Logging with Serilog and MongoDB

This page describes structured logging, and how to use it with your JavaScript logs.

A working demo is in project JSNLogDemo_Serilog_MongoDB.

About structured JavaScript logging

When logging an event, often you not only want to log a string, but also the values of some variables, or even an entire object. However, in most cases the log entries are stored as simple strings, such as in a text file or in a column in a relational database.

This leads to code like this:

log.Fatal(string.Format("Something happened! x={0}, y={1}", x, y))

This is fine until you want to search or order log entries based on those variable values. If you want to find all log entries with x < 34, sorted descending by the values of y, you'll have to somehow parse your log entries.

Instead of destroying the structure of your log entries by converting them to strings, and then resurrecting that structure by parsing them, why not leave the structure in place? This is where structured logging comes in.

Structured JavaScript logging from your server side application

Whereas logging packages such as Log4Net and NLog store your log entries as flat strings, structured logging packages such as Serilog allow you to preserve the structure of your log entries.

For example, Serilog allows you to write this C# code:

var position = new { Latitude = 25, Longitude = 134 };
var elapsedMs = 34;
log.Information("Processed {@Position} in {Elapsed} ms.", position, elapsedMs);

It will then not only log the string Processed { Latitude: 25, Longitude: 134 } in 034 ms., but also the individual values of position and elapsedMs.

Storing structured logs

How would you store a structured log? Text files are out, because you don't want to destructure log entries into flat text anymore. Relational databases are not practicle, because you would have to define columns for each variable you would ever want to log.

However, NoSQL document databases are ideal for this task. Unlike relational databases, these allow you to store objects of any shape, without having to pre-define a schema. However, they offer the same ability as relational databases to search and order objects based on their properties.

MongoDB is a popular document databases that is supported by Serilog. In the remainder of this page you'll see how to use it to store your structured JavaScript logs.

Set up structured JavaScript logging

Now that you've seen what structured logging is, you'll see how to use JSNLog and Serilog to log JavaScript events in a structured way in a server side MongoDB database.

Step 1: Install MongoDB

Download and install MongoDB;
Add the installation directory to your Path system variable.
- Windows 7: right click Computer | Properties | Advanced System Settings | Environment Variables | Under System Variables select Path | Edit
- Windows 10: Windows key | Type Advanced System Settings | Environment Variables | Under System Variables select Path | Edit
At the time of writing, the installation directory was C:\Program Files\MongoDB\Server\3.2\bin
Create directory for your MongoDB database:
```
C:\data\db
```
Open a new command prompt and start MongoDB:
```
mongod
```

Step 2: Install Serilog

First install Serilog and the Serilog sink for MongoDB with the Package Manager Console:

Install-Package Serilog.Sinks.MongoDB

Then configure Serilog. Serilog doesn't support configuration in your web.config, it has to be done in code before you do any logging. A good place to do this would be in your Global.asax.cs file, in the Application_Start handler:

using Serilog;
...
protected void Application_Start()
{
    ...

    // Write to database logs of the MongoDB instance running on localhost 
    var log = new LoggerConfiguration()
                    .WriteTo.MongoDB("mongodb://localhost/logs", period: TimeSpan.Zero)
                    .CreateLogger();
    Log.Logger = log;
}

You can now log events from your C# (or Visual Basic) code using Serilog (how).

Note that unlike SQL Server, there is no need to create a database. It is created on the fly when you write your first log entry.

Serilog is very configurable. The documentation gives you some pointers but for more details you have to read code comments.

Step 3: Install JSNLog

Install JSNLog and the adapter to Serilog with the Package Manager Console:

Install-Package JSNLog.Serilog

Access your structured logs

Once you have done some logging, access the MongoDB database to access the log entries:

Open a second command prompt and start a MongoDB shell:
```
mongo
```
(do not close the command prompt running MongoDB itself that you opened earlier)
Switch to the logs database:
```
use logs
```
The records in a database are organized in collections. Your log entries will have been written to the log collection. To list all records in that collection:
```
db.log.find()
```
db.log.find() doesn't show you all records in one go, but breaks the output into pages. To go to the next page, type:
```
it
```
To only see the actual log information, filter out all properties except for the property called Properties:
```
db.log.find( undefined, {Properties: 1, _id: 0})
```
Visit the pages below to see how to filter and sort records.

Find out more: