Job System Enhancements - Version 2's Got Em!

Summer means vacations and pool time, but we haven't stopped working on Exceptionless 2.0. Things are coming along nicely, and today we're here to talk about the job system and the code being written to enhance it.

After you read this article, check out the previous V2.0 feature and detail articles, if you haven't already. Good stuff in there!

  1. Exceptionless 2.0 – In the Making
  2. Event Based Reporting System
  3. Simplified API
  4. A Pluggable System
  5. Exceptionless 2.0 Client Rewrite Sneak Peak and Example
  6. New Message Bus and Queuing System
  7. Job System Enhancements

Job System Enhancements #

Standalone #

Jobs can easily be run standalone now, which makes it much easier to test the system. You won't have to worry about your application pool shutting down prematurely and killing your job half way through it's long-running work item.

More Ways to Run #

With 2.0, you'll be able to run jobs in process, as a service, as a standalone exe, or as Azure WebJobs. Cool, huh?

Need More? Run More! #

If you need to process more tasks, simply fire up more jobs. This will save resources and money when using Azure WebJobs, as you can auto-scale jobs based on resource constraints.

We use jobs to send emails, call web hooks, process events, and much more, so these enhancements will allow for better resource control when scaling, and in general.

Questions? #

We've thrown out a lot of information about Exceptionless 2.0 and all the new features, enhancements, and tweaks that it's going to get. We'd love to hear from some of the regulars out there and see if we've missed anything obvious. Check out the links to the other articles at the top of the page and let us know. Thanks!

Version 2.0's New Message Bus and Queueing Systems

In an effort to improve scalability, allow for new functionality to easily be added to Exceptionless, make the system less coupled, process things more efficiently, go fully Async, and further support Azure, we've been working hard on a new message bus and queueing system.

Lets take a look at a few of the details surrounding these new systems we're building for Exceptionless 2.0. Take a look and let us know what you think. If you've got questions or comments, we'd love to hear them!

Message Bus #

The new message bus allows us to publish and subscribe to messages across all our resources. We subscribe to different types of messages to send SignalR notifications to the client.

Queueing #

The new queueing system allows us to enqueue expensive tasks that can be handled at a later time. This lets us greatly reduce the processing and latency times of the api.

  • We stream event data that is posted to the event controller directly into the queue without taking the IO or Memory hit of processing it. This means that we can process more errors, faster, with less resources.

The system also supports retrying and discarding of data.

  • We queue emails that need to be sent, as well as user defined webhooks that need to be called with data. Email servers on the sending and receiving can go offline or error out while sending, but by queuing the notification emails we can ensure you always get them by re-sending in the future, after a failure occurs. In the event that we can't send you an email after a few retries, we can discard the notification.

More on Version 2.0 #

If you're just now learning about the upcoming Exceptionless 2.0, make sure to catch up on previous feature announcements and examples by reading the below articles.

Exceptionless 2.0 Client Rewrite Sneak Peek Usage Example

new-client-header

As Exceptionless 2.0 continues to become a reality, we thought we would give everyone a little taste of what you will be able to do with the new, rewritten client. Continue reading for a glimpse at the primary features, along with a complete usage example for adding extra data to events.

After you check it out, let us know if you have questions or suggestions. We're listening!

New Client Features #

  • The Exceptionless client has been completely rewritten to be highly simplified and extensible.
  • Will work with Mono and Project K.
  • The base client is PCL, and we will have platform specific clients that add additional functionality for each platform.
  • Adding extra data to events is extremely easy.

View Client Source

Extended event data usage example #

<img loading="lazy" class="aligncenter size-large wp-image-9100" src="/assets/img/news/ex-client-1024x420.png" alt="Exceptionless Code Example" width="940" height="385" data-id="9100" srcset="/.png 1024w, /assets/img/ex-client-300x123/nt.png 1167w" sizes="(max-width: 94/

Lets break that example down, line by line, shall we? Check out the client source if you want to take a look at the complete code.

First, set your API key.

var client = new ExceptionlessClient(config => {
config.ApiKey = "API_KEY_HERE";

Then, send events to your own free Exceptionless server install.

config.ServerUrl = "https://exceptionless.myorg.com";

Now, read config settings from attributes.

config.ReadFromAttributes();

Read config settings from a config section in your app/web.config.

config.ReadFromConfigSection();

Store all client data including the offline queue in the store folder, by default isolated storage is used.

config.UseFolderStorage("store");

Exclude any form fields, cookies, query string parameters, and custom data properties containing "CreditCard".

config.AddDataExclusions("CreditCard");

Add the "SomeTag" to all events.

config.DefaultTags.Add("SomeTag");

Add the "MyObject" custom data object to every event.

config.DefaultData.Add("MyObject", new { MyProperty = "Value1" });

Add a custom event enrichment that will add a tag called "MyTag" to every event.

config.AddEnrichment(ev => ev.Tags.Add("MyTag"));

Register a custom log implementation that uses NLog.

config.Resolver.Register(new NLogExceptionlessLog());

The Startup method is specific for each platform and wires up to all relevant unhandled exception events so that they will be automatically sent to the server.

client.Startup();

Manually catch and report an error with a custom tag on it.

try {
throw new ApplicationException("Boom!");
} catch (Exception ex) {
ex.ToExceptionless().AddTags("MyTag").Submit();
}

Let users add their email address and a description of the error.

client.UpdateUserEmailAndDescription(client.GetLastReferenceId(), "me@me.com", "It broke!");

Create and submit a log message and add an extra "Order" object to the event.

client.CreateLog("Order", "New order created.")
.AddObject(new { Total = 14.95 }, name: "Order")
.Submit();

Submit a feature usage event that will let you see how much certain features of your app are being used.

client.SubmitFeatureUsage("FeatureA");

Submit a page not found event so you can keep track of your broken links and fix them.

client.SubmitNotFound("/badpage");

Listen to all events being sent and cancel any errors that are "IgnoredType".

client.SubmittingEvent += (sender, args) =>
args.Cancel = args.Event.IsError() && args.Event.GetError().Type.Contains("IgnoredType");

Settings data is synced in real-time with the project settings in your Exceptionless project on the server.

client.Configuration.Settings.Changed += (sender, args) =>
Trace.WriteLine(String.Format("Action: {0} Key: {1} Value: {2}", args.Action, args.Item.Key, args.Item.Value));

You can use those settings to control behavior in your app.

if (client.Configuration.Settings.GetBoolean("IncludeMyCustomData", false))
Trace.WriteLine("Should include my custom data");

That's All There Is To It! #

After checking out the above example, we hope you agree that we've drastically simplified and improved the process of adding data to events, allowing for much more flexibility.

As always, if you have any questions, comments, suggestions, or concerns, let us know!

Read more about Exceptionless 2.0 #

Coming in Exceptionless 2.0 - A Pluggable System

Pluggable System

In the last Exceptionless 2.0 article, we announced the upcoming simplified API. Today, we want to introduce another major piece of V2.0 - the pluggable system.

Plugins will allow customization and translation throughout the Exceptionless platform, including integration with third-party services and more. Read on for more details about pluggable details such as event parsing, event pipeline, and formatting.

Event Parsing #

  • Has access to the raw POST data as well as the content type and submission client info.
  • Used to translate that raw data into events.
  • Can easily create new plugins to support new data formats like system logs.
  • Can be used to support other JSON formats like adding support for clients made for other systems.

Source

Event Processor #

  • Can be used to add new functionality to the system.
  • Gets called on startup, when an event is starting to be processed and when an event is done being processed.
  • Has access to settings from both the org and project level.
  • Can be used to create integrations for 3rd party services like HipChat, Trello, GitHub, Slack, etc.

Source

Formatting #

  • Used to control how events are displayed in the system.
  • Controls the summary view of an event.
  • Controls the stack title.
  • Controls what notification emails look like.
  • Controls which view is used to display the details of an event.

Source

We believe building a pluggable exception reporting system and allowing third-party service and app access will create one of the most flexible, usable, and friendly solutions on the market.

Coming Soon #

We're anxious to get Exceptionless 2.0 wrapped up, but we do not have an ETA currently. We are working hard and making good progress, so keep an eye out for more sneak peeks, feature announcements, and progress reports!

As always, please let us know if you have any feedback or questions.

More from the Upcoming Exceptionless 2.0: Simplified API

Exceptionless 2.0 API SimplifiedSince going open source, we've wanted to simplify the API and make it easier to work with.

We're taking the time to do it now, and it's going to be awesome!

Exceptionless 2.0, coming soon, will have a new, manageable API with tons of great documentation and examples. Take a look at the preliminary documentation at the below link, and make sure to give us any feedback you might have.

API Simplified #

  • New REST API documentation and samples site.
    Take a look and let us know what you thinkExceptionless API Documentation
  • Event POSTs take the raw data and use a plugin system to interpret that data and translate them into events.
    • This allows us to take literally any data and turn it into events in the system.
    • The POST data is captured as a raw bytes and added immediately added to a queue for processing.
    • Plugins can easily be created to support new data formats like system logs.
  • This simplified API will make creating libraries for other platforms dead simple.
  • The API lives in a separate project and can be hosted on high-performance systems like the new Helios IIS host.
  • Makes it easy for us to migrate the UI to a SPA app.
  • Now uses OAuth 2.0 in addition to supporting API tokens.
  • Highly consistent REST API modeled after GitHub and Stripe.
  • It's so simple you can just use CURL as a client.

We hope you're as excited as we are to have this new, improved, more complete, and more usable documentation. Stay tuned for more details on the upcoming Exceptionless 2.0, and don't forget to leave a comment letting us know what you think.

Exceptionless 1.5 Released!

Exceptionless Version 1.5While we're on the march to Exceptionless 2.0, we're still making updates and fixing bugs on version 1. Today, we'd like to announce that Exceptionless 1.5 has been released, which includes several server changes and bug fixes, as well as major client code base optimization.

Please update your client to version 1.5 and take a look at the other changes and bug fixes, below. We've done quite a bit of work to notifications, added throttling to improve coverage on small and free plans, and improved performance in a few places.

Server Changes #

  • Added throttling to accounts that are over their usage limits. If an account is sending a high number of errors, the errors will be throttled on an hourly basis so that the entire plan limit won’t be used up immediately. This allows for a distributed sampling of the errors instead of only capturing everything in a short period of time.
  • Added a site notification that shows you when error submissions are being throttled or if you are over your monthly plan limits.
  • Removed total count from most recent errors list as it was a very expensive to calculate while providing little value.
  • Fixed a bug with notifications that could cause some users to get spammed. Now notifications only send a maximum of 10 notifications per project every 30 minutes.
  • Greatly simplified the authentication logic for the web api pipeline.
  • Added the ability to print all content on the error occurrence page.
  • The pager will no longer scroll to the top of the current list when changing pages.
  • Updated the paged lists to only refresh the list data via push notifications when you are on the first page.
  • The list data will only be updated in real time if the data matches the current filter criteria.
  • Fixed a bug where the loading indicators would appear on the suspended and manage organization pages.
  • Fixed a bug where the save button on the manage organization page would have improper styling.
  • Fixed a bug where a HttpAntiForgeryException could be thrown when accessing the website.
  • Fixed a bug where a ArgumentException would be thrown if multiple model validation errors occurred on a single page.
  • Fixed a bug where a NullReferenceException could be thrown when signing up.
  • Added some additional checks to try and resolve the user profile when an invited user signs up.
  • Fixed a bug where an updated organization notification could be sent before the user was authorized to access the organization.
  • Fixed a bug where empty OS Name and Version values were being shown in the errors environment section even if they didn't exist. This could happen if the client was reporting from an azure website instance.
  • Changed billing plans to use per month error limits.
  • Fixed a bug where the BillingManager could throw a NullReferenceException for a newly added organization. This could happen because the primary node had not replicated the content to the secondary nodes or the data wasn't cached on creation.
  • Updated various MongoDB collections to not persist empty array fields.
  • Fixed a bug where some cache entries were not automatically expiring.

Client Changes #

It's highly recommend that you update your clients to 1.5 as we did major optimizations to the client code base.

  • Greatly simplified how the client processes and sends errors. The client now properly handles the various status codes that can be returned from the service.
  • Added an event that allows you to customize the request object before it is sent to the service.

As always, please let us know if you have any questions!

Event Based Reporting System Coming in Version 2.0

Event Based Reporting SystemWe hinted that more details on the upcoming Exceptionless 2.0 release would get announced soon, and here we are! Lets dive in a bit further, shall we?

Many users have asked for ways to use Exceptionless to report additional types of events, rather than just errors. With version 2.0, we are moving to an event based system that will accommodate such requests.

What's an Event Based Real-Time Reporting Tool Look Like? #

  • The new system allows us to receive literally any data people want to send us instead of only allowing errors.
  • Event posts can be as simple as this:
    Post Event Exceptionless
  • You can send log messages or even entire log files.
  • Log messages can contain extended data objects just like errors can now.
  • You can post random JSON objects and the data within them will be treated as extended data.
  • You can post batches of events instead of only being able to send one at a time.
  • You can send feature usage events that let you see how often features of your application are being used. Think about how useful that will be!
  • You can send session start and end events that will enable you to know what percentage of users are affected by errors and enable you to better know what your priorities should be.
  • We will be gathering enough data to make it easy for us to begin putting together some very useful analytic reports.

We're pretty excited about the switch from error-only to send-us-any-event-you-can-think-of real-time reporting, logging, and notifications. We think it's going to be awesome, and it's almost scary how much of a playground Exceptionless is going to turn into for some of our customers. We're not pushing the limits, we're pushing for no limits!

Ideas? Concerns? Let us know. We're working hard to wrap up Exceptionless 2.0, but there's still a lot more bells and whistles we're polishing before launch! Keep an eye out for still more sneak peek material in the coming weeks!

Exceptionless 2.0 - In the Making

Exceptionless Version 2.0 Sneak PeekIt may seem quiet in Exceptionless land, but the truth is we've been writing, and re-writing, more code than you can point a cursor at. If it weren't a labor of love, our fingers would have mutinied long ago, but luckily they are in it for the long haul and are churning out some seriously sweet new features and rewrites.

Exceptionless 2.0 will include many of the feature requests that have come in since we launched, and will drastically expand on the current functionality. We know you'll love it, so continue reading for a high level view of what's coming in the near future.

Event Based System #

Many users have asked for ways to use Exceptionless to report additional types of events, rather than just errors. With 2.0, we are moving to an event based system that will accommodate this.

API Simplified #

Since going open source, we've wanted to simplify the API and make it easier to work with. We're taking the time to do it now, and it's going to be awesome. Watch out!

Pluggable System #

Plugins will allow customization and translation throughout the Exceptionless platform, including integration with third-party services and more.

Client Rewrite #

The client is being re-written be highly simplified and extensible, opening up platform support and ease of functionality additions.

Coming Soon #

Version 2.0 is coming soon. In the mean time, look for more details and sneak peeks containing examples of functionality and usage. We can't wait to show the world!

Don't forget to follow along with the development on GitHub and watch Exceptionless 2.0 while it's being built!

Web Application Errors and the 80-20 Rule

Most Frequent ErrorsYou know the 80-20 rule. The one marketers, managers, bigwigs, writers, speakers, and everyone else references to tell you what to focus on or why the widgets aren't selling like they should.

Well, as painful as it might be to hear the rule applied to yet another scenario, we're here today to let you know that 80-20 can be applied, in many cases, to your code's exceptions as well. 

20% of Errors Cause 80% of Exception Instances #

That's not so bad, is it?

All we're trying to point out is that when you start tracking and organizing the errors in your code, inevitably you'll start to realize that a small percentage of bugs are causing the majority of your total errors.

What we did about it #

We realized this ourselves, so we built the "Most Frequent Errors" report right into the Exceptionless dashboard.

When you log into your account, you immediately see which errors are causing the largest number of exceptions, how many instances of each there is, when the first occurrence was, and when the last occurrence was.

From there, you can click in, view there error's details, and hopefully find a quick resolution to the problem.

Most Frequent <> Most Important #

While it's great to chip away at the most frequent errors, it's worth pointing out that most frequent doesn't always mean most important, so be sure to focus on bottom line, customer facing, and major functionality issues first!

Have You Observed the 80-20 in the Wild? #

In your coding adventures, have you found the 80-20 rule to apply to your exceptions? Was it something you found quickly, or did you track down numerous instances of the same error before realizing it?

A Few Recent Exceptionless Case Studies

Error Stack Dashboard

We like to know how Exceptionless is helping its users, and we love it when we get feedback! Every once in a while, we like to share a story or two so we can drive home the benefits of having a real-time exception logging service tied into code projects.

Surveys are sometimes sent out to users, and we wanted to share a few recent responses.

Exceptionless Case Study Questions #

  • What is the number one customer-facing bug that Exceptionless has helped you track down? Can you give us a brief description of the bug and how you solved it?

  • User 1: "Missing DLLs in the client environment, due to required 3rd party apps not being installed (eg: Crystal Reports runtime). [Exceptionless] enabled me to create a list of missing DLLs and computers for relay to the client."

  • User 2: They were previously handling their own exception logging well and don't have any serious customer-facing errors coming in.

  • When you first started using Exceptionless, were you surprised by the number of errors that were being reported? Approximately how many were you seeing per day?

  • User 1: "Not really, but just because I had written a similar tool that logged these before, they just were not as accessible (database table on web service)." The user reported that the above bug was appearing 10-15 times each day.

  • User 2: They were already handling their error logging and already had errors down to a minimum, around 50 per month. "The primary reasons that we switched to Exceptionless from our own code is that Exceptionless captures additional detail (including code line numbers), Exceptionless has a better interface and design for reviewing and managing errors and bugs (including summary views that we didn’t previously have), and we no longer need to maintain our own error-logging code."

  • What is the number one internal bug that Exceptionless has helped you track down? Can you give us a brief description of the bug and how you solved it?

  • User 1: N/A

  • User 2: "...a small number of situations where our code doesn’t correctly handle empty record sets (that is, where, based on the specific query string parameters in the URL, no corresponding records are found in the database and the page is expecting that there will always be records to display)."

  • What is the number one feature request or change that you would like to see the Exceptionless Team tackle?

  • User 1: "More options to filter dashboard by tags/versions/environment variables. Custom reporting, exporting to CSV file etc or other way to import them into TFS or similar."

  • User 2: "...my number one request is probably for additional information in daily summary emails and for a combined daily summary. Notification defaults and moving notification settings out of the project setup is probably a close second; or perhaps the ability to add comments when marking an issue as fixed would be my second highest feature request."

  • If you have any other other examples of Exceptionless helping you squash bugs, please share them below - we would love to hear them!

  • User 1: N/A

  • User 2: "...in our small team, the additional information from Exceptionless (as well as the process of converting to Exceptionless) has prompted us to review and solve a few ongoing bugs that we were aware of..."

What has Exceptionless Done for You? #

If you use Exceptionless to log, report, organize, and squash errors in your code, we want to hear how it helps, what kind of crazy errors you are eliminating, and your general thoughts on the project! Don't be shy, we won't bite.

Thanks to Philip with the Beth Israel Deaconess Medical Center and Michael with Customer Logic for their case study contributions. We appreciate it guys!