We recently introduced reverse geocoding into Exceptionless and are now adding features to make full use of it.
What we'd like to do in this blog article is walk any interested developers through the process of adding it to their own app.
We'll talk about the resources and services we're using to pull it off, why we chose them, and give you code snippets for implementation. It's all open source, so we've also included links to all the relevant code in hopes it will make your life easier!
Wouldn’t it be nice if you could provide location services to your users automatically? Maybe help them fill in a shipping form from a zip code or there current location?
With the launch of Exceptionless 2.0 we added a geo property to all events. This allows us to translate valid latitude and longitude coordinates into a physical location. Our goal was to begin capturing the data then and enable different scenarios and uses later. This also allowed us to show end users where their customers events are being submitted from.
One of our primary goals with Exceptionless is to be completely open source and easy to use (both in setting up self hosting and using the product). We had to take this into account when picking any library or service, because we want a painless setup and no additional costs for self hosters, all while adding additional value!
Please note that if you love the services we use, you should look into using one of their paid plans or at least promoting them with a positive review, shout out, etc (everyone needs to eat at the end of the day, right?).
After researching many different services, we ended up goin
g with GeoLite2's free, offline, downloadable databases. These databases are free and updated once a month, but if you require a more accurate and up-to-date database they offer a paid subscription. We also use their open source library for interacting with the database in memory.
We use our very own Foundatio Jobs to download the most up-to-date database. Foundatio Jobs allows us to run the job in process or out of process on a schedule in Azure.
Alternatively, you could use the PowerShell script we created for downloading the database. <a href="https://github.com/exceptionless/Exceptionless/blob/master/src/Exceptionless.Core/Jobs/DownloadGeoIPDatabaseJob.cs" target="_blank">DownloadGeoIPDatabaseJob</a> downloads the database over http and extracts the file contents to disk using Foundatio Storage.
Please feel free to take a look out our job for a complete sample including logging and error handling:
var client =newHttpClient(); var file =await client.GetAsync("http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz", context.CancellationToken); if(!file.IsSuccessStatusCode) thrownewException("Unable to download GeoIP database.");
Looking up a Physical Address from an IP Address #
Resolving the geo coordinates into a location
After we automate the database download, the next step involves loading the database in memory using the open source library provided by MaxMind and querying by the IP address. The code below will do very basic IP validation and lookup the records using the API.
var database =awaitGetDatabaseAsync(cancellationToken); if(database ==null) returnnull;
try{ var city = database.City(ip); if(city?.Location !=null){ returnnewGeoResult{ Latitude = city.Location.Latitude, Longitude = city.Location.Longitude, Country = city.Country.IsoCode, Level1 = city.MostSpecificSubdivision.IsoCode, Locality = city.City.Name }; } }catch(Exception){ // The result was not found in the database }
if(!await _storage.ExistsAsync("GeoLite2-City.mmdb")){ Logger.Warn().Message("No GeoIP database was found.").Write(); returnnull; }
try{ using(var stream =await _storage.GetFileStreamAsync("GeoLite2-City.mmdb", cancellationToken)) _database =newDatabaseReader(stream); }catch(Exception){ // Unable to open GeoIP database. }
return _database; }
Then, just call the ResolveIPAsync method with an IP address to look up the location details.
var location =awaitResolveIPAsync("YOUR_IP_ADDRESS_HERE");
Feel free to take a look at <a href="https://github.com/exceptionless/Exceptionless/blob/master/src/Exceptionless.Insulation/Geo/MaxMindGeoIpService.cs" target="_blank">MaxMindGeoIPService</a> for a complete sample that includes logging, error handling, caching of the results, and IP validation for higher lookup throughput. We’ve spent the time writing tests and optimizing it to ensure its rock solid and works great. So feel free to grab our IGeoIPService interfaces and models and use them in your app.
It’s worth noting that in our app, we use the IP address provided in the event. This could come from a server request or the actual machine's IP address. We also fall back to the API consumer's client IP address.
Looking up a Physical Address from Geo Coordinates #
As stated previously, every event submitted to Exceptionless has a geo property that can be set. If it’s set, we will attempt to look up your location by the geo coordinates using a third party service. We used the open source Geocoding.net library, which abstracts the major different third party reverse geocode services into an easy to use API (options are always good!).
After we decided on the library, we evaluated a few API/lookup services based on cost and accuracy. We ended up going with the Google Maps GeoCoding API. They offer 2500 free requests per day and are one of the most used location services in the world.
Next, let’s write the code that will look up our location from a latitude and longitude. You can find our complete example here.
Remember to get your free api key from Google before running the code below.
publicasyncTask<GeoResult>ReverseGeocodeAsync(double latitude,double longitude,CancellationToken cancellationToken =newCancellationToken()){ var geocoder =newGoogleGeocoder("YOUR_API_KEY"); var addresses =await geocoder.ReverseGeocodeAsync(latitude, longitude, cancellationToken); var address = addresses.FirstOrDefault(); if(address ==null) returnnull;
It took us a bit of work and research initially to get everything working flawlessly for location services. We hope you grab our code off of GitHub to save yourself all that work. Also, it’s worth noting that we use Foundatio Caching to cache the results of location lookups. It drastically increased the performance and cut down on our limited number of requests to rate-limited third party services!
We also queue work items to look up the geo location since that can be an expensive operation. So, please take this into consideration anytime you are interacting with a third party service.
Get in touch on GitHub or leave a comment below to let us know your thoughts or questions. We're always open to suggestions and will do what we can to help you out if you're having issues with implementation!
Today we bring you a case study from the founder of Referral Rock and serial entrepreneur, Joshua Ho.
Referral Rock is a referral platform for small businesses that Josh created after he "... realized small businesses lacked the tools to make a customer referral program work." The app allows businesses to easily and effectively create, implement, and administer a rock-solid referral program to help grow their business.
Exceptionless recently became a part of Referral Rock's exception reporting and logging toolkit, replacing Joshua's home-grown exception logging solutions, and here are his thoughts!
"
I've always done my own exception logging. Very basic stuff, where I would just log exceptions to my local drive. This gave me a nice place to just look at errors in my ASP.NET code. As with most code, it ended up in production deployments. At one point, I even built some web pages to view the logs remotely. Those little exception logging code snippets also made it into about 3-5 other projects as the years went by. I knew there was software out there that could do this, but I more or less had it solved for myself.
**But that changed recently.**
"One huge benefit Exceptionless adds to my business is giving me the ability to provide better customer support."
As I've been growing my own SaaS business for referral software, called Referral Rock, I realized there were times my old solution wasn't effectively capturing all the exceptions and I would have to venture into the Event Log to find out what happened. Also, I liked being able to view the production server remotely, but my little web pages got lost somewhere and I wasn't excited about coding them again. Who likes to code the same thing more than once? Not I.
"I could see details on the requests, browser cookies, user identity... all much more than I could see using my old solution."
So that led me to look for other solutions to view my exceptions remotely, which is when I found Exceptionless. With the help of support, I got it up and running fairly quickly. The guys at Exceptionless were very responsive and helpful in getting me setup.
Being a startup, I was initially using the free version for about 2 weeks and was blown away. The UI was great and I love the AngularJS based responsiveness. Soon I had a great pulse on my production server. I could see details on the requests, browser cookies, user identity... all much more than I could see using my old solution. Once I had it set up, I started to see the benefits using other types of events in Exceptionless, such as logging. I started adding some logs when I was debugging an issue with a customer, and it worked great.
"With the help of support, I got it up and running fairly quickly. The guys at Exceptionless were very responsive and helpful in getting me setup."
One huge benefit Exceptionless adds to my business is giving me the ability to provide better customer support. Not only do I know when errors are happening, but also who is seeing them. This allows me to have an opportunity to reach out to that specific customer, once the issue is fixed, and say something like "I saw you had and error when you did XYZ, I wanted to let you know it is fixed now so you can try it again". Taking opportunities to provide that level of service has helped my business.
We are now running a paid version of Exceptionless with multiple projects and I look forward to adding more logs and playing with other features to give me even greater visibility into my web app. Thanks guys!
We love to see people enjoying Exceptionless - it's our baby, and we've put a lot of blood, sweat, and tears (I blame Blake) into it. Keep rocking it with Referral Rock!
To many, this feature may be the missing piece... that connection you've always wanted to make between users, bugs, exceptions, app events, etc. I'm talking about, of course, user session tracking!
That's right, you can now use Exceptionless to track users as they use your app, which of course will come in super handy when you want to know exactly what they did to stumble upon or cause an error or trigger an event.
Continue reading to learn more about sessions and how you can enable them for your apps.
First, you must have a paid (premium) Exceptionless plan to report on sessions if you are hosting with us. This is mainly because of the added resource requirements the feature puts on our infrastructure. We think it's definitely worth it, though!
Sessions can be searched/filtered like all other events - they are just an event type like exceptions or logs.
Each user session records how long they were active and what they did. For instance, the average user session on be.exceptionless.io the first week we monitored it using the feature was two hours.
With that, each user session event has a "Session Events" tab that displays all the relevant events that occurred during that session and allows you to see exactly what that user did. This is especially helpful, of course, if that session lead to an exception or noteworthy event in your app.
First, you'll need to update to the latest client versions to enable sessions, then you'll have to follow the below steps to begin tracking them. Once you've got that set up, visit the new Sessions section under the Reports option on your main dashboard, or navigate directly to https://be.exceptionless.io/session/dashboard. If you are self hosting, make sure you update to Exceptionless 3.2 first.
For Exceptionless to track a user for your app, you need to send a user identity with each event. To do so, you need to set the default user identity via the following client methods:
Once the user is set on the config object, it will be applied to all future events.
Please Note: In WinForms and WPF applications, a plugin will automatically set the default user to the **Environment.UserName** if the default user hasn't been already set. Likewise, if you are in a web environment, we will set the default user to the request principal's identity if the default user hasn't already been set.
If you are using WinForms, WPF, or a Browser App, you can enable sessions by calling the EnableSessions extension method.
Sessions are created in two different ways. Either the client can send a session start event, or we can create it automatically on the server side when an event is processed.
We have a server-side plugin that runs as part of our pipeline process for every event - its sole purpose is to manage sessions by using a hash on the user's identity as a lookup for the session id.
If the session doesnt' exist or the current event is a session event type, a new session id will be created. If we receive a sessionend event, we close that session and update the end time on the sessionstart event.
We also have a CloseInactiveSessionsJob event that runs every few minutes to close sessions that haven't been updated in a set period of time. This allows you to efficiently show who is online and offline during a time window.
How do I Enable Near Real-Time Online/Offline Then? #
We do this by default in our JavaScript, WinForms, and WPF clients when you call the UseSessions() method.
In the background, we send a heartbeat event every 30 seconds if no other events have been sent in the last 30 seconds.
You can disable this heartbeat from being sent by passing false as an argument to the UseSessions() method.
The WinForms and WPF clients will also send a SessionEnd event when the process exits.
Can I Manually Send SessionStart, SessionEnd, and heartbeat Events? #
Sure! You can send these events manually via our client API to start, update, or end a session. Please remember, though, that a user identity must be set.
As always, please send us your feedback. You can post it here in the comments or submit a GitHub Issue and we will get back to you as soon as possible! We're always looking for contributors, as well, so don't be afraid to jump in and be the hero the code needs. Contributors get Exceptionless perks!
That's right folks - we've gone and released Exceptionless 3.2, which includes releases for Exceptionless.NET, Exceptionless.JavaScript, and Exceptionless.UI! Awe yeah.
We're kind of excited, in case you couldn't tell. Big stuff in here, like session tracking (#BOOM), licensing changes (less confusion - it's a good thing), and posting via HTTP GET (such easy, much wow)!
Track and automatically manage user sessions for much more visibility into their user experience, how they interact with your app, and, of course, any errors or events that occur related to that user. This answers the age-old question, "What the hell was this guy doing when stuff blew up!?"
Now it's even easier to integrate with Exceptionless from any environment, because you can post event or meta data via HTTP GET! More info coming soon (blog post).
The server and all Exceptionless projects are now using the Apache License, so there should be much less confusion on how things are licensed moving forward. Boring stuff, we know... but important.
User locations are now resolved from geographic coordinates or the IP address. We look at the geo property for coordinates or an IP, then we inspect the IP. If no IP or geo coordinates present themsevles, we fall back to the client IP that the event was submitted from.
As always, we keep speed improvements in mind with each release. With 3.2, we've been able to make more massive improvements in processing time for events (over 250% per 1000 events!) and further reduce app startup times and elastic query execution times. #alwaysoptimizing!
The hourly event-throttling threshold has been increased from 5x to 10x the plan limit. The way we calculate it is by taking the plan limit and dividing it by the hours in the month, then multiplying it by 10.
User experience was the primary focus of this UI release, along with the new sessions feature. More details below, including other improvements and a few bug fix details.
When adding a new project, users will now have a much better experience, and we added a JavaScript configuration section for JS projects. Check it out!
Besides integrating with the above, the only major change in the JavaScript client, like the .NET client, was that users can now op-in to sessions. To do so, set a default user and call the below:
In the case study, we discuss how Elasticsearch let us improve our real-time dashboard, work with time zones more easily, index data much faster, not have to worry so much about pre-aggregating data, maintain much less code, improve backup and restoration, shrink disk space usage, and, perhaps most importantly, drastically improve scalability.
Basically, we really like Elasticsearch and you should check it out if you haven't already.
This week we want to talk about reference Ids and how we use them to improve customer service for Exceptionless. And how we think you can do the same for your users.
A reference id is a unique identifier, generated by the user, that allows you to look up a submitted event at a later time. This is important because event Ids are not created until after the event is processed, so there is no other way to look up an event. We also use Reference Ids to help deduplicate events server side, which is another reason why it’s important that they are unique.
One of the ways we use Reference Ids is to support end users, something that is very important to us. There is nothing more frustrating than being unable to help a user that is experiencing a problem because you don’t know what they are seeing (when there could be thousands of errors in your app).
To combat this issue, we always try to include a reference Id with every error message shown to the user. This allows end users to contact us with a Reference Id and receive help immediately for the error they are seeing because we can easily track down and view it.
The Exceptionless clients have built in support to generate and track unique Reference Ids for every error event. Behind the scenes, we register a default Reference Id plugin that sets a Reference Id on submitted error events and stores the Reference Id in an implementation of ILastReferenceIdManager. The Reference Id manager’s goal is just to persist and retrieve the last used Reference Id.
Since this is a default plugin, we can enable this behavior by calling the UseReferenceIds() method on the configuration object to register the default Reference Id on every error event.
Please note that you can create your own plugin to create your very own Reference Id(s).
To get the the last used Reference Id, you can call the GetLastReferenceId() helper method on the the ExceptionlessClient instance.
C#
usingExceptionless; // Get the last created Reference Id ExceptionlessClient.Default.GetLastReferenceId();
JavaScript
// Get the last created Reference Id exceptionless.ExceptionlessClient.default.getLastReferenceId();
You might have noticed how easy it is to get or add Reference Id’s to your events automatically. This makes it a breeze to let your developers track down user-facing issues by displaying the Reference Id to your end users.
We display Reference Ids to all of our end users anytime an error occurs in our ASP.NET WebAPI application. We accomplish this by adding a custom <a href="http://www.asp.net/web-api/overview/error-handling/web-api-global-error-handling" target="_blank">IExceptionHandler</a> and return a new error response to include the Reference Id as shown below:
var exceptionContext = context.ExceptionContext; var request = exceptionContext.Request; if(request ==null) thrownewArgumentException($"{typeof(ExceptionContext).Name}.{"Request"} must not be null",nameof(context));
// CreateErrorResponse should never fail, even if there is no configuration associated with the request // In that case, use the default HttpConfiguration to con-neg the response media type if(configuration ==null){ using(HttpConfiguration defaultConfig =newHttpConfiguration()){ return request.CreateResponse(statusCode, error, defaultConfig); } }
You might be thinking: "Reference Ids are great, but what do I do with them now that I have one." Well, you can view the event that they reference on our site or via our API. This can be accomplished two different ways:
Hotlinking: You can link directly to a submitted event by outputting a link in your UI or logs (e.g. https://be.exceptionless.io/event/by-ref/YOUR_REFERENCE_ID)
Search: You can search via our api/ui with reference:YOUR_REFERENCE_ID
We hope this article was helpful, and we'd love to know if you're using Reference Ids and how they've helped you help users, solve internal issues, etc. Post a comment below!
Prior to version 3.1, we used an early version of the Foundatio Jobs system to run our out-of-process jobs via Azure WebJobs. We found it to be quite a pain to figure out which jobs were running or eating up system resources because every job was titled Job.exe (just like figuring out the w3wp IIS process is running). Also, just to run an out-of-process job, one would have to compile the source, copy dependencies to a common bin folder, and then run an executable (Job.exe) with parameters that specify the job type.
These tedious and error-prone tasks that had to be completed just to get a job to run are a thing of the past.
Now, with Foundatio, our open source app building block solution used in Exceptionless, you just define a new Job that runs (via the run method) and you can use the Foundatio Jobs API to run the job in process, out of process, continuous, or one time without changing the implementation.
This new approach also gave us a great deployment strategy, for free. Simply copy the job executable and bin folders and run it anywhere!
Jobs (processes) running in Azure as an Azure web job #
Foundatio Jobs allows you to run a long running process (in process or out of process) with out worrying about it being terminated prematurely. By using Foundatio Jobs you gain all of the following features without changing your job implementation:
Run job in process
Run job out of process
Run job with a start up delay
Run job in an continuous loop with an optional interval delay.
In this sample we'll just define a new class called HelloWorldJob that will hold our job that increments a counter and derives from JobBase. Please note that there are a few different base classes you can derive from based on your use case.
But our goal is to run this out of process in an Azure WebJob (this also works if you want to run this as a service or from the desktop).
The first step is to create a new console application and reference the Foundatio NuGet Package and the project that contains our HelloWorldJob. We are going to call our console application HelloWorldJob. Inside of the Program class, we'll update the main method to run our job.
namespaceHelloWorldJob{ publicclassProgram{ publicstaticintMain(string[] args){ // NOTE: This should be the path to your App_Data folder of your website. var path = Path.Combine(AppDomain.CurrentDomain.BaseDirectory,@"..\..\..\Api\App_Data"); if(Directory.Exists(path)) AppDomain.CurrentDomain.SetData("DataDirectory", path);
// Get a service provider so we can create an instance of our job. var serviceProvider = ServiceProvider.GetServiceProvider("JobSample.JobBootstrappedServiceProvider,JobSample");
var job = serviceProvider.GetService<JobSample.HelloWorldJob>(); returnnewJobRunner(job,initialDelay: TimeSpan.FromSeconds(2),interval: TimeSpan.Zero).RunInConsole(); } } }
The last steps are to simply compile the project and deploy it to your Azure website!
Frank's back at it, adding support for collecting extra exception data that was already in the .NET client like module info, custom exception properties, and everything else that was already displayed in the UI when using the .NET client.
Read Configuration
We added support for reading configuration from environmental variables and app settings
**Closing Applications
** Closing applications after submission is now easier due to the new SubmittedEvent event handler.
**Custom Persistence Settings
** The new QueueMaxAttempts and QueueMaxAge configuration settings allow custom persistence settings and are intended to improve offline support. Thanks @airwareic!
Data Exclusions Improvements
We've made huge improvements to Data Exclusions, which now check default data and extra exception properties.
**New Overloads
** Thanks @xbelt for creating overloads for CreateLog and SubmitLog to accept the LogLevel enum!
Custom Themes
@mgnslndh updated the styling of the CrashReportDialog to play nice with custom themes. Thanks!
**Dependencies
** All dependencies (Nancy, NLog, etc) have been updated.
**Deprecated!
** The EnableSSL property has been marked Obsolete because it is no longer used. ServerURL is now being looked at for this.
If you thought Exceptionless was fast before, prepare to have your mind blown by what we've been able to do in version 3.1, which released today.
In short, we've reduced cpu and memory usage, increase caching efficiency, and sped up searching, all quite significantly as you'll see below.
Along with these speed boosts, we've also made Job improvements (look for a blog post on this soon) and upgraded to .NET 4.6 (self-hosters, please install .NET 4.6 on your servers before upgrading).
Details on the release changes can be found below.
We reduced the CPU and Memory load across the entire app. This allows the application to use fewer resources, meaning it has more time to process additional events, making everything faster. Between making Exceptionless and Foundatio 100% Async and these improvements, we've drastically increased the efficiency of the entire platform.
Below, we see the increase in performance from two examples. On the left, we see a reduction in CPU and Memory usage for a deployed web app instance. On the right is a visible reduction in CPU usage for an Elasticsearch node.
By profiling the Elasticsearch queries for efficiency and usage, we've been able to reduce the overall number we were running and improve the efficiency on the ones that still are.
Caching efficiency has been improved by removing redundant components that were utilizing valuable resources. For example, we removed the SignalR Redis Backplane, which drastically decreased the number of calls to Redis. Overall, we've made the app smarter throughout regarding how we cache and retrieve data.
We've offloaded long-running API tasks to background jobs, freeing up a lot of resources in the app and allowing us to scale work items independently. For example, marking a stack as fixed or removing a project may take a few moments to be updated now, but the trade-off is worth it. We're working on updating the UI experience to prompt users that the task is running in the background.
Exceptionless is now running on .NET 4.6, which has improved startup time due to various improvements with the new version. Self-hosting users should be sure to upgrade to .NET 4.6 on their servers before updating Exceptionless.
We're always striving to improve the efficiency of Exceptionless and all of our projects. If you see any room for improvement or have any comments when using anything from us, please send us an in-app message, submit a GitHub issue or contact us on the website.
First, you'll need to update to the latest client versions to enable sessions, then you'll have to follow the below steps to begin tracking them. Once you've got that set up, visit the new Sessions section under the Reports option on your main dashboard, or navigate directly to https://be.exceptionless.io/session/dashboard. If you are self hosting, make sure you 
That's right folks - we've gone and released Exceptionless 3.2, which includes releases for Exceptionless.NET, Exceptionless.JavaScript, and Exceptionless.UI! Awe yeah.
We've got a quick release for the Exceptionless JavaScript Client that includes a few new additions to what it supports and a quick bug fix or two.
The .NET Exceptionless client has received several improvements and some bug fixes in version 3.2. Details below!
If you thought Exceptionless was fast before, prepare to have your mind blown by what we've been able to do in version 3.1, which released today.
](/assets/img/news/sessions-event-tab-user-footsteps.jpg){kind=link}
](/assets/img/news/sessions-unique-user-data.jpg){kind=link}