Weekly Update - Bug Fixes, Archival Format Changes, and Foundatio Updates

This week we've got some cool stuff in our weekly update! Along with the below, you can now follow along over on the new YouTube Exceptionless Weekly Updates playlist, so make sure to check it out and subscribe!

Let's Get Down To Business #

Updates to Exceptionless #

  • We fixed an issue where you couldn't run Exceptionless on AWS because you couldn't install plugins on their ElasticSearch service. #280

  • The archival format has also been change from projected\year\month\day to year\month\day\hour\projectid. This allows you to quickly restore or download all events in a time period without enumerating all backed up projects.

Foundatio Updates #

  • In Foundatio, we fixed an issue where Redis Queues could deadlock, causing them to stop processing. This was being caused from topic messages locking the calling thread.

Updates to Foundatio Repositories #

  • GetById and GetByIds now take an Id Type which allows us to specify routing information and much more. We have an implicit conversion that handles existing signatures.


Watch more Live Code Demos

Email Logging, UI Improvements, Foundatio Updates, and more - Live Code Demo

Exceptionless live code demo 2/20/17

Watch out this week, Blake's on fire! We're talking email loggings, UI tweaks, Exceptionless.NET updates and fixes, Foundatio updates, and Foundatio.Repositories updates. Lot's going on, let's check it out in this week's Live Code Demo.

Exceptionless Updates #

This week, we made improvements to email logging and documentation when running in dev mode. Find out more by watching the live coding video or visiting the Exceptionless repo on GitHub.

Exceptionless.UI #

We were only showing the exception tab if there was an event type of error, previously, but now we are always showing the exception tab if an exception is part of an event.

View Repo

Exceptionless.NET #

  • We added SetException EventBuilder overload so you can submit any event type with an exception object.
  • Then, we fixed an issue where the client could blow up on startup while trying to wire up to trace listeners.
  • Finally, we also fixed an issue where signed web packages http handlers weren't being registered with the right namespace.

View Repo

Foundatio #

  • Attempted to track down issues where the redis queues would stop processing.
  • Worked with Microsoft to get our unit tests discoverable on the new csproject format.
  • Added new Foundatio.Jobs commands package that allows you to quickly discover and get command line help on your jobs.

View Repo

Foundatio.Repositories #

  • Fixed various issues with running data migration scripts for the first time.
  • Fixed an issues with GetByIds where falling back to search wouldn't take into account multiple pages of results.

View Repo


Watch more Live Code Demos

Duplicate Event Total Fix, Setting Min Log Levels, Foundatio Updates, and more - Live Code Demo

exceptionless duplicate event totals, min log levels, foundatio

Blake's back at it this week, onward to something new after the Exceptionless 4.0 launch. Today we talk deduplicate event totals, setting minimum log levels, updates to Foundatio, and tracking down an event processor job issue!

  • We had previously reverted to earlier commits that showed the deduplicated event total separately from the event total, but since we are running on new indexes now, we are able to reliably include the deduplicated total as the total number of events. GitHub Issue #270

  • You can now set the min log level in configuration using:


    Learn more

  • The Foundatio updates we made include performance and handling improvements to the message bus. Learn more about these improvements on GitHub. We also added IHybridCacheClient interface marker, giving you granular control over dependency injection. Now, you can inject either a cache client or a hybrid cache client much easier. Learn more.

  • Lastly for this week, we are working on tracking down the root cause of an issue with the event processor job that is causing it to stop processing events. More on that soon!


Watch more Live Code Demos

Live Code Demo & Review of Exceptionless 4.0 Launch

During the Exceptionless 4.0 Release, we covered a lot of enhancements and changes. In this week's Live Code Demo, Blake reviews the 4.0 release notes and talks through other tweaks and post-launch cleanup.

In this Live Code Demo Episode... #

  • Exceptionless 4.0 has been released for a few weeks now, with ElasticSearch 5 running in production for even longer. Let's take another look at the release notes and talk through some notable topics.
  • We look at the self hosting & upgrading documentation, which has been updated for Exceptionless 4.0.
  • In the process of the latest release, we upgraded Foundatio to new csproject format, logging many bugs on the .NET CLI along the way. Blake reviews this in the above video as well.
  • In the end, we obviously had to merge in pull requests and clean up issues, which he also reviews.
  • Lastly, we cleaned up the new Azure infrastructure!


Watch more Live Code Demos

Live Code Demo - New Query Validation, DI Issues, and Error Handling

This week's live code demo comes to you during a week of final testing and tweaks prior to our launch of Exceptionless 4.0.

Blake dives into and shows you the code for several important improvements and fixes, which we've listed below.

  • We've added a much improved Query Validator to the repositories and Exceptionless.

  • Some DI issues with running jobs under the debugger have also been fixed.

  • Then, we added error handling around heartbeat submission end points.

  • And finally, an issue where the app would blow up when trying to save invalid reference ids has been fixed.

Watch Video

Watch more Live Code Demos

Exceptionless 4.0 Enhancement Overview - Live Code Demo

Exceptionless 4 Enhancement Overview

We recently announced Exceptionless 4.0, but we also recorded a short demo of some of the enhancements and talked through them.

In this live code/feature/functionality review video, Blake talks about:

  • Deploying Elasticsearch 5.x to production with completely new Azure resources.

  • Fixing bugs and doing extensive testing on the new Elasticsearch 5 and Azure implementation.

  • Massive performance boosts due to daily indexes, SSD-backed storage, and bug fixes. These boosts should result in near0instant page loads and huge improvements to session dashboards.

  • Automating cluster setup using Azure CLI and Elasticsearch ARM Templates.

  • A new custom SSH script that allows us to quickly and securely connect to machines, as well as forward ports for things like Kibana.

  • New Azure resources and dashboards.

  • Setting up WebJobs to run as cron jobs.

  • We also found PM2, a new way to manage Node.js services for our StatsD service.


Watch more Live Code Demos

Exceptionless 4.0 is Here!

Exceptionless 4.0 AnnouncementThat's right folks, Exceptionless 4.0 is here and we've got the full low-down on all the improvements, upgrades, and enhancements in this week's blog article, along with how to upgrade instructions for self hosters.

The primary focus of version 4.0 was to add support for Elasticsearch 5. By migrating from Elasticsearch 1.7 to 5, we removed growing technical debt and benefited greatly in the process. Here is an overview of the benefits gained by migrating to Elasticsearch 5 in 4.0.0.

Performance #

Since we first migrated to Elasticsearch 1.x from MongoDB, we've learned a lot of hard lessons. With 4.0 and the upgrade to Elasticsearch 5, we took all those lessons and applied them to the new release and infrastructure setup. Here are some performance improvements and notes that came out of those implementations.

  • We've moved from monthly indexes to daily indexes. This change means that when you are filtering by the last 24 hours or last week, you are greatly reducing the amount of work Elasticsearch has to do to return the requested data.

  • Because of this, the average use case will see between 25% and 80% improvement to indexing throughput. This means that, with the same hardware, we can index the same documents much faster!

  • We moved from Groovy scripts to Painless scripts, which are four times faster. By moving to Painless scripts, we also greatly simplified the burden of having to modify configuration files to enable scripting!

  • Elasticsearch has added a lot of new data types since 1.7, and we are now taking advantage of them to reduce memory, storage and query costs. This means we can query the same data faster, using less memory doing it. We've also set up more sensible defaults to ensure we don't index very long strings.

  • We've also made several performance and reliability improvements to snapshots (backups).

Self Hosting improvements #

Our goal is to have everyone be able to setup and use Exceptionless in a matter of minutes. One of the areas that had hendered a lot of self hosters in the past was forgetting to update the elasticsearch.yml file, which is no longer an issue with the migration to Painless scripts. If you're a self hoster, check out the

  • The move to using Painless scripts as part of bulk updates and ingest pipelines reduced the burden of having to modify configuration files to enable scripting! Long gone are the times where you would have to reset your setup because you missed a configuration step.

  • We've also added various maintenance jobs that handle backups and restores automatically, so you don't have to worry about losing your data!

  • Future upgrades should be seamless as Elasticsearch now handles reindexing out of the box! Once you have migrated your data to Elasticsearch 5, we think future major upgrades will just be handled by the Exceptionless app itself!

  • You can now also use Docker or Azure arm templates to quickly set up a cluster. We really like this direction and will continue moving down this path, hopefully getting to the point where you can click a single button and have an Exceptionless production instance running locally!

Dashboards #

With Exceptionless 4.0.0, we continued to focus on finishing up backend improvements to both our repositories and parsers that we made in the 3.5 releases. We feel that all the pieces are finally in place to allow us to do custom dashboards in the near future, something we talk about in our 2017 Roadmap blog post.

Foundatio.Parsers now gives us the ability to define and validate custom aggregations using Lucene-style syntax. For example, lets say I want a date histogram that shows the min, max and average event value. I'd just need to pass date:(date min:value max:value avg:value) to the following endpoint: /api/v2/events/count?aggregations=date:(date min:value max:value avg:value) to return aggregations across a time series!

Upgrading to 4.0.0 #

The only users that need to worry about upgrading anything for this new release are self hosters. If you are self hosting Exceptionless, please review the Self Hosting Documentation, which contains information about upgrading your existing install.

Also, please take a look at the change log for a full list of the changes.

Always Improving #

We’re always striving to improve the efficiency of Exceptionless and all of our projects. If you see any room for improvement or have any comments when using anything from us, please send us an in-app message, submit a GitHub issue or contact us on the website.

And Lastly, Thanks! #

We'd like to say thank you to the community, project sponsors, and the Elasticsearch team for helping us ship 4.0.0. You guys rock!

Want to know what's coming next? #

Check out the 2017 Roadmap and let us know your thoughts on our planned development.

Setting Up New Elasticsearch Cluster, Reindexing & Mappings - Live Code Demo

Getting Ready for Exceptionless 4.0! #

In this live code review, we go over some of the recent changes and processes to get ready for Exceptionless 4.0, which include implementing a new Elasticsearch cluster, reindexing into it using the Elastic Arm templates, and updating existing events mapping.

The Elastic Arm templates automate the setup of an Elasticsearch with a few clicks, which is awesome.

Then, we had to update our existing events mapping to have a last updated field and is deleted flag so we can do incremental reindex passes.


Watch more Live Code Demos

2017 Exceptionless Feature, Functionality, and Enhancement Roadmap

exceptionless feature roadmap for 2017

Can you believe it's almost February already! Us either. No rest for the whicked, though, as we sit down to get cranking on our list of Exceptionless changes for 2017.

We have a a great list of features, functionality changes, and enhancements planned, and no doubt we'll work in a few other popular feature requests and, naturally, bug fixes along the way.

Let's take a look at the primary roadmap, shall we?

Upgrading to Elasticsearch 5 #

Upgrading to Elasticsearch 5 has been in the works for some time now, with early testing starting in late 2016. As of January 23rd, we have officially pushed the upgrade to production and are looking for feedback, missed bugs, and performance improvement reports from anyone using it!

Primary Elasticsearch 5 Benefits #

  • Elasticsearch 5 is the latest version of ElasticSearch and brings in massive improvements to event indexing speed, reduced memory sizes, and much more.
  • The Elasticsearch libraries support .NET Core, bringing us one step closer to fulfilling a vision that includes cross-platform support.
  • Our new implementation is built on a new, faster Microsoft Azure and SSD storage infrastructure. This will greatly increase the indexing and searching performance,  reducing dashboard latencies, among other things.
  • With the power of Foundatio.Parsers project, we have gained the ability to consume generic aggregations and pivot/report on data any way our users like it.
  • This upgrade also makes it easier to self host through the use of containers. Our goal is to move the entire app to use containers, which will allow you to self host in seconds!
  • We will now be doing daily indexes for more performance, fine grained backups, and the ability to more quickly change our indexes.

Future Exceptionless Notification Changes #

We're always working on notifications, we've got several distinct changes we want to work on in 2017. Here are a few of the important ones:

  • We will continue working on rich, rule based notifications for email, services like Slack and HipChat, and more.
  • Pausing notifications will soon be a thing!
  • You'll also be able to set up notifications based on the rate of events, so you can know when issues are ramping up or getting out of hand.
  • Lastly, you will be able to receive a periodic digest email based on activity.

Porting to .NET Core and Containers #

  • We want to port the Exceptionless app to .NET Core so we can run it anywhere (macOS, Linux or Windows).
  • Running on .NET Core also brings massive performance improvements and lower overhead, which will have positive trickle down affects throughout the app.
  • Once we support .NET Core, we can containerize the entire app, allowing us to scale more easily.
  • These changes will make self hosting super simple. You'll basically be able to run "docker run -d exceptionless`" and be done!

Custom Exceptionless Dashboards #

We know the dashboard is crucial for many things, and we want to make them more customizable to fit each user's individual cases. At the same time, we want to make sure we maximize business value. So, we plan on working on several features and functional aspects of dashboards, namely:

  • Creating pre-define dashboards that are more granular, such as device usage, breakdown by exception type, etc.
  • Allow users to create their own custom dashboards.

New Exceptionless Clients #

We really want to add new native clients to Exceptionless, and are including that as a major goal for 2017. Everything we are working on is moving in that direction, one way or the other, and we hope to make further announcements in this department soon!

We are, of course, always accepting pull requests as well!

What Are We Missing? #

What have we forgotten? We know everything isn't included on this list, but if we have missed something big that you can't live without, please let us know!

Otherwise, look for a new version announcement very soon, and get ready to upgrade and take advantage of several of these new upcoming changes.

Upgrading from Elasticsearch 1.x to 5.x - Live Coding Session

elasticsearch 1.x to 1.5 upgrade

New Weekly Coding Live Stream #

In the first episode our all new weekly live streaming demo session, Blake talks about the recent push to migrate Exceptionless from Elasticsearch 1.x to Elasticsearch 5.x.

One major part of this project has been updating Foundatio.Repositories and Foundatio.parsers, both of which are parts of our pluggable foundation block project for building distributed apps, to support Elasticsearch 5.x.

Once that was complete, we then worked on reindex scripts for migrating data from Elasticsearch 1.x to 5.x using Elasticsearch's external reindex.

Check out the live stream, and tune in weekly for the next episode.

Want Blake to demo something specific? Just leave a comment here and we'll make sure he gets it done!


Watch more Live Code Demos