That's right! We've re-imagined how session heartbeats and session end events should work on the back end and were able to make them much more efficient, allowing us to stop counting them toward user plan limits!
This blog post explains our original goals and implementation of these session events, and how we were able to retain the same functionality of the feature while limiting resource usage.
Our hope is that this will obviously make our users happy, but also that all the developers out there can benefit from our process and solution.
In the beginning, we set out to create a sessions feature that allowed our users to submit a session start event and the session would be automatically updated by sending a session heartbeat event, as well as a session end event, respectively.
These session heartbeats and session end events were meant to be session markers to show that a user was active or a session had ended.
We wanted to leverage our existing infrastructure, and the easiest way to do so was to introduce new event types that we recognized. This meant that these events went through the client side plugins (extra work) and server side processing. As such, there was no way to tell these events apart from any other event.
And because of that, they counted against user plan limits.
After releasing the sessions feature, it didn't take us long to notice that the heartbeats were noise. However, we knew that users wanted to see what their customers or users were doing while being active throughout the session, so we didn't remove them.
We knew they were counting towards plan limits, causing some users to reach theirs quickly, and we knew they were adding noise, thus limiting the feature's value, but we also wanted to keep the feature alive because of the potential value offered. So, we had to react and make the entire feature more feasible, streamlined, and cheap.
We did some thinking (and coding), trying to determine the best way we could provide end users with a great session tracking feature without over-taxing our system in the process, and we were able to come up with a solution!
So, we created a new GET API endpoint /api/v2/events/session/heartbeat (api source) that takes a session id or user id and a flag if the session is closed. This API endpoint then sets a unique session cache key with the current time.
Our existing CloseInactiveSessionsJob.cs was already periodically polling for open sessions to check for inactive sessions so it could automatically close them after a period of time if no session end event was sent, so we just updated this job to check for the unique session cache keys (source) and get the last time a heartbeat was sent in or see if it was closed. It then takes the appropriate action and updates the session event.
Then we just updated the clients to call this new API endpoint when await client.SubmitSessionHeartbeatAsync("id") or await client.SubmitSessionEndAsync("id") is called.
Our new solution gives us the ability to have clients send us session heartbeat and session end information very efficiently, which lets us provide a great session tracking feature without adding any additional cost to our plans!
That's how we like to roll, and we hope you find value from the feature and our run down of the process we went through to get it to our users.
Exceptionless.NET v3.5.0 and Exceptionless.JavaScript v1.4.0 have been released and they are chock full of new features and feature improvements!
There weren't a whole lot of bug fixes with these releases. Could it be that Exceptionless is working and we've squelched most of the bugs? We think so!
Notably, we focused on sessions, real time configuration, and logging, all of which we think you'll find super helpful.
Session Heartbeats No Longer Count Towards Plan Limits! #
We've updated Session Heartbeats and Session End events to be sent through an optimized API end point, and they will no longer count towards your plane's event limits.
You can now exclude events by type and source, along with setting minimum log levels, in real time. So, for example, if you accidentally disable logging completely, you can simply update your settings and the updates will go into affect, restoring functionality!
UI support will be expanded moving forward, but for the time being here is an example project configuration.
Source
Key:@@EVENT_TYPE:SOURCE<br />
Value:false
Any event type or source can be used, and wildcards * are supported, so you can ignore whole event types and set the source to *.
Disable Submissions of Events with Type Name
Key:@@error:*MyCustomException
**Value: **false
This configuration will disable submission of any error event with a stack trace containing an exception with a type name that ends with MyCustomException. You could also remove the wildcard and specify the full type name with the namespace.
Log Message Settings
Key:@@log:*
Value:off
Even though log types are special cased, they still accept rue or false values, so this example completely turns off log messages.
Other known values are Trace, Debug, Info, Warn, Error, Fatal, and Off.
Minimum Log Level
**Key: **@@log:MyNamespace.EventPostsJob
**Value: **Info
This example is setting the minimum log level for EventPostsJob to Info.
Automatically Check for Updated Configuration Settings on Client Idle #
We've implemented an automatic recurring check for updated configuration settings that occurs two minutes after the last event submission.
A few notes:
Each configuration check does not count towards your account's plan limits.
No user information will be sent - only the current configuration version.
Nothing will be retrieved if no settings have been changed.
The automatic recurring configuration settings check can be disable by calling:
New Easy Way to Exclude Events from Being Submitted in .NET #
You can now define a simple Func<Event,bool> callback to stop events from being submitted in the .NET client.
For example, if I wanted to ignore any event with a value property of 2 I could use client.Configuration.AddEventExclusion(e => e.Value.GetValueOrDefault() == 2);
Bug Fix for SettingsCollection Boolean Values Support in .NET Client #
SettingsCollection.GetBoolean(name) was not supporting `` or 1 as boolean values, but now it does. We are also now supporting yes and no as valid values.
We hope these new features and feature improvements help your Exceptionless experience, and we plan to continue to listen to user feedback and improve the system where improvements need to be made. To do that, though, we need to hear what you want out of the app. If you have a favorite feature request, or something that just bothers you, please let us know by dropping us a line on GitHub Issues under the appropriate repository, listed below, or just comment here and we'll figure it out!
Today we've got a quick article on one of the little known features of Exceptionless: keyboard shortcuts.
In an effort to help you navigate Exceptionless event stacks better and troubleshoot faster, we've added keyboard shortcuts that lets you do things like go to the stack from an individual event occurrence, copy the event JSON, navigate to the previous event occurrence, etc.
Let's take a look at all the quick shortcuts you can use to make your Exceptionless usage more efficient.
To bring up the keyboard shortcuts available to each page, simply hold the shift key and hit the forward slash key (SHIFT + /, think ?). This will bring up an overlay showing you which shortcuts are available on that specific page.
Don't see shortcuts available on a page? Don't freak out, there may just not be any. Feel free to suggest something that you think would save yourself and others time, though!
If you hit SHIFT + / (?) on an event occurrence, you will see three or four new options, depending on whether you are on the first occurrence in the stack or not.
**Go to Stack
** On any event occurrence, you can hit CTRL + UP to go to that event's stack quicker than finding the "Go To Stack" button and clicking.
**Copy Event JSON to Clipboard
** Hit CTRL + SHIFT + C to copy that event's JSON data to your clipboard.
**Previous Occurrence
** CTRL + BACK (LEFT) will take you to the previous event occurrence.
**Next Occurrence
** And CTRL + FORWARD (RIGHT) will take you to the next occurrence of that event.
On any dashboard or stack, SHIFT + / (?) brings up a simple overlay letting you know that ? will close the overlay and F1 will take you to the Exceptionless documentation on GitHub.
We're open to suggestions for other dashboard or stack shortcuts, just let us know in the comments!
We'd love to get some community input here. What shortcuts do you think would be useful on which Exceptionless pages? Do you have something you do repeatedly on a dashboard or stack page that you think a keyboard shortcut might speed up or make more efficient? Let us know in the comments and we'll do our best to implement your ideas!
Prioritizing your bug fixes and development time in general can be a daunting task.
Sometimes, as developers, we want to work on this shiny widget or this annoying bug, and we don't really have anything in our face telling us to quit focusing on our dreams and work on what matters to the bottom line.
I can hear you over there: "But, my dreams are important!" Well, yes, but you don't get to have fun working on those until you've made your users happy by fixing the bugs that are affecting the majority of them or expanding on that feature that they are all using every single time they use your app.
We've got something that will let you get those pesky tasks off your plate though, so you can move on to the fun stuff!
Our new "users" column and "Most Users" dashboard lets you know exactly what percentage of your users are being affected by events or using features. This allows you to prioritize the most important bugs or features to work on right away and potentially backlog things that only a few users are having issues with or using.
Of course, you'll need to be sending at least a user id (and preferably a display name) for each user. We'll cover how to do that later in the article.
The new Users column on your Exceptionless dashboard displays a percentage value for each event stack that represents the number of users that have been affected by the event or, if it is a feature, that have used the feature.
If you mouse over the percentage, you can see the number of users the percentage represents out of the total.
These numbers are dynamically calculated for your selected timeframe that you are currently viewing.
Because the main dashboard shows you the most frequent events, not necessarily with the highest usage, we thought it would be helpful to have a new dashboard that automatically sorts event stacks by the percentage of users affected, letting you start at the top of an exception list, for example, and work your way down knowing you're always working on a bug, etc, that is affecting the highest percentage of users.
In order to assure you are getting value out of the user feature, you want to make sure you are setting the user. For Exceptionless to track a user for your app, you need to send a user identity with each event. To do so, you need to set the default user identity via the following client methods:
Once the user is set on the config object, it will be applied to all future events.
Please Note: In WinForms and WPF applications, a plugin will automatically set the default user to the **Environment.UserName** if the default user hasn't been already set. Likewise, if you are in a web environment, we will set the default user to the request principal's identity if the default user hasn't already been set.
You can also manually set the user info on the event directly. This is intended for **multi-user processes (web applications). **For most MVC and WebAPI packages, the user will be set automatically based on the logged in principal, so you don't have to do anything.
// Import the exceptionless namespace. usingExceptionless; ExceptionlessClient.Default.CreateFeatureUsage("MyFeature").SetUserIdentity("123456789","Blake Niemyjski").Submit();
If you're using the JavaScript client, the entire session of the client will typically be for a single user, so you should be able to set it one time when they log in to your app.
We are always interested in how our users use our features, and if our users feature helps our users help their users, well, that's a win win for our users and their users. Go users!
Don't forget to stop by and let us know if you love or hate it, and of course let us know if you think we can improve on anything within Exceptionless.
We've continued to focus some of our attention on our pluggable app foundation block solution, Foundatio, and last week it got some attention from the Microsoft .NET Blog as well!
We've got a few new implementations with this release, including Amazon S3 storage, Azure Service Bus queue (thanks @jamie94bc!), and Redis metrics client. Users should find those helpful.
The APIs have been drastically simplified for ease of use and straightforwardness, which should allow for easier creation of new implementations and consumption of existing ones.
Once example of this simplification is that we've paired down jobs to having only one lock, instead of four.
Logging has also seen huge improvements to make sure we are aligned with future API implementations, etc.
We always work hard to keep improving Exceptionless, and this release is no different!
Since the last release, we've put a lot of time into making the filtering and searching more user friendly and intuitive, improving the reliability of jobs, and of course fixing any bugs that you guys (or Exceptionless) have been able to find.
For starters, as mentioned above, we relocated the search bar to exist on the top level of the UI, and the date picker filter now shows the current choice on the top level. Both icons were replaced, and we really think it's much more intuitive and efficient. You can read more and see examples over on the dedicated blog post we did last week.
Session management has been drastically improved by doing a few different things. For instance, inactive sessions are now closed faster, but they can be opened again if need be. We hid heartbeat events by default, too, and you can now specify manual sessions for desktop-based apps.
We introduced custom event stacking a few weeks ago, and @adamzolotarev has added the ability to specify a manual stacking key on the client side with this release. Thanks Adam!
A default list of bot wild card exclusions is now automatically set on new projects, so if you're upgrading, you now have the ability to run a maintenance job via the admin controller to set a default bot list. All events with user agents matching these wild cards will then be discarded on the client side.
Marking stacks as fixed or hidden was causing some significant slow down and sometimes wouldn't work at all. This has been remedied!
Redis connection failures and lock timeouts were sometimes causing jobs to stop working or fail. We dug through and found what was causing that and fixed it as well.
When the geo field contained a localized number, sometimes events were not being processed. This localization issue has been solved.
And last but not least (well, maybe least), a seralization bug has been fixed that would cause query strings, cookies, and other extended data items to be transformed to lowercase and underscored.
Well, only if you're a self hoster. Everyone else will experience all of these awesome improvements and bug fixes the next time they log in. If you are a self hoster, please review the Self Hosting Docs for info regarding upgrading your current Exceptionless install. Naturally, if you have any questions please let us know and we will get you taken care of.
You can find a complete comparison changelog over on GitHub, where you can also submit any issues, etc if you run across anything. Please also let us know what you think of the changes by commenting below, pinging us on social media, or simply sending Blake a glitter bomb(site/link not endorsed in any way, lol - first one I found!).
It's been a while since we introduced filtering and searching when we launched Exceptionless V2.0, so we decided recently that we wanted to take the feedback we've received and do a round of improvements to it.
You may have already noticed the changes, but if not then the next time you log in you will see that the top bar has changed, giving you much quicker access to filtering and more upfront information.
For the primary desktop view, we removed the magnifying glass icon in the top bar and simply filled the rest the bar with the filter (search) input box. This eliminates a click to get to the filtering system, and keeps it front and center at all times.
One important note here is that if you want to show events that have been marked as fixed or hidden, you have to explicitly specify those filters, whereas previously those options were check boxes. So, you can use hidden:false or hidden:true, and fixed:true or fixed:false to view those events. Naturally, the default is false for both, showing events that have not been hidden or fixed. This means that in order to see both hidden and un-hidden events, you would need to use hidden:false OR hidden:true. Likewise, for fixed, you would need to use fixed:true OR fixed:false.
You'll notice that the date/time filter has changed as well. Instead of an icon, we now display the current time frame being viewed, once again saving you a click and keeping things in front of you.
As always, the filter still applies to the chosen time frame only.
We also changed the user interface for smaller screens.
Instead of the icons at the top of the screen, the time frame selector is now a major menu item in the mobile menu and displays the current selection with the filter/search field directly below it.
This setup should allow users to filter and find the events they seek much quicker on mobile.
We think it's a pretty nice improvement. We got feedback from several users and think making everything visible at the top level of the user interface is an important change that saves time and keeps you informed.
If you've got any additional feedback, please don't hesitate to let us know. We are always looking for ways to improve, and we use Exceptionless every day too, so we are always interested in saving ourselves time and making things easier on our end!
We’ve learned a lot about simple app deployment since we first started Exceptionless. We initially went with what everyone else was doing (Octopus Deploy), but over time we thought we could greatly simplify and automate it, letting us focus on what matters, improving the product!
Through a lot of testing and iterations of our deployment process, we think we finally nailed it.
As such, we’d like to share with the community how we use Microsoft Azure Continuous Deployment and GitHub for **awesomely simple deployments. **And, how you can too. See the details of implementing this deploy workflow later in the article, below.
When we first started Exceptionless, we deployed it as a monolithic application, meaning the server and UI were one piece, or app. We used Octopus Deploy to deploy to a single IIS machine, which involved setting up a website and server for the Octopus Deploy service and configuring build agents, on each server we deployed to, that could deploy build artifacts.
After a year or so of managing Exceptionless on colocation boxes in a Dallas data center, we realized that we didn’t want to manage hardware anymore and we could scale easier on a managed service like Azure. So, we moved to Azure, where we had to set up a VM just to manage deployments with Octopus Deploy. There were also issues that we ran into with deploying to Azure WebSitThis was annoying, since every time we wanted to do a release we had to log in and tell the system to deploy to production.
Soon, we decided to split the UI and Server apps so we could deploy and work on them independently. This also meant they could scale independently and one change to either wouldn’t cause the whole site to go down when deploying. Splitting the two helped a lot, but it added more work as we now had to manage two Octopus Deploy projects. So, we started looking at the Continuous Deployment in Azure.
We researched further and found out that if we used Git Flow as a development workflow, we could ditch Octopus Deploy completely, remove that dependency, and just use Git push to manage our deployments.
With Git Flow, you do all your work in feature branches and the master branch is always stable. This allows us to set up GitHub deployment on the master and deploy to Azure automatically, with no work required! So anytime we push to the GitHub master branch, it automatically deploys to production on Azure. That simple!
Here is the BASIC workflow:
We create a new feature branch, then work on that branch until it is completed and tested. Testing is done on the website that is currently pointed to the feature branch, which is separate from production.
When we commit to any branch, our continuous integration (CI) server picks up the changes via a GitHub webhook, pulls them down, then builds the project.
We then take all of the build artifacts and push them to a second GitHub repository using the same branch that the code was pushed to (for example, the master branch). This allows you to see exactly what artifacts change between releases (stateless too) and different branches.
Those changes are then automatically pushed via Azure Git Deploy.
Profit!
This is very slick! Since we push artifacts to the same branch they were built onto a build repository. We can then set up different environments that get auto deployed when we push to that branch. For example: When we are working on a feature, we commit to our branch. We can then set up a new website in Azure that pulls from the build server’s Git artifacts branch for that feature. This allows us to test and automate environments!
2. Store build artifacts in a separate GitHub artifacts repository. #
This works really well because you can see the entire history of your build artifacts and browse their contents. Plus, GitHub hosts it for free!
We found that we could that GitHub could understand and parse into different links. We can click on the "Commit:" part of the message to link to the actual commit that is building to see exactly has changed.
We can then click on the build to see what artifacts changed.
Another great thing about using Git to store your artifacts is that you can easily clone the artifacts to your local machine to see the exact files that are being used in a specific environment.
The artifacts repository has branches to match the branches of our code repo so we have separate build artifacts for each branch. This also means that we can just merge the feature into master when we are done and that will cause the production website that is pointed to our master repository to automatically get updated. So, it’s as simple as merge your branch to master to promote a build to production.
One issue with this approach is that the repo can get large because we are storing binary files that change on every build. We are looking into using Git Large File Support to fix this issue.
3. Automate pushing of artifacts to a secondary GitHub repository. #
For our .NET application, Exceptionless, we invoke a PowerShell script on post build to clone and commit the changes to the Git artifacts repository.
For our static Angular JavaScript app (UI), we invoke a Grunt publish task from our post build event. The publish task called into a gh-pagestask that publishes our built dist folder to the GitHub artifacts repository automatically.
4. Point Azure Continuous Deployment to the Artifacts Repository #
It will see when new artifact commits happen and automatically deploy the changes.
Azure Continuous Deployment is another Git repository that we can easily view to see the history of deployments to each of our sites. It also allows us to easily roll back to previous versions.
5. Use Environment Variables to Override Config Settings Per Environment #
Azure Websites makes this very easy.
No production settings are stored in source control or artifacts repository.
For our ASP.NET application, our settings class will look up named settings in the following order:
environment variables
local config.json file
app settings
It will then fall back to a default value if no explicit settings are found.
Configuring our static Angular JavaScript app is a bit more work since it can't look these settings up dynamically at runtime. So instead we add some code to our deployment process.
Azure automatically runs a deploy.sh file after getting the artifacts via git deploy. It’s sole job is to run a node script that rewrites our app.config.js settings with settings defined in environment variables.
You can create multiple Azure websites (think environments) that use Continuous Deployment and point them to multiple artifact branches to support different environments!
Pro Tip: We created a http://local-app.exceptionless.io website for our spa website that’s pre-built and points to your localhost Exceptionless server. This allows us to do work on the server part without setting up or configuring a local instance of our spa app. Development made simple!
We won't lie, it took some work to get here, but the good news is you can do this really easily too. Please feel free to steal our deployment scripts and modify them for your projects. And let us know if you have questions along the way!
We get it... your morning coffee, folded towels, and how events stack (group) in your event reporting application should be controllable and customizable.
Well, thanks to a great suggestion by @adamzolotarev, now they are! Well, the events, at least.
We do our best to group your events into relevant and smartly-named stacks, but there are cases where you may want to specifically name a stack and attribute certain events to it for organization, reporting, troubleshooting, or other reasons.
To facilitate this need, we created SetManualStackingKey, which both .NET and JavaScript client users can set.
Adding your own custom stacking to events in Exceptionless is super easy. Below are examples for both .NET and JavaScript.
In these examples, we are using setManualStackingKey and naming the custom stack "MyCustomStackingKey".
So, any events you use the below for will be a part of the custom stack, and all other events, exceptiones, logs, feature usages, etc will still be stacked automatically, like normal, by the app.
try{ thrownewApplicationException("Unable to create order from quote."); }catch(Exception ex){ ex.ToExceptionless().SetManualStackingKey("MyCustomStackingKey").Submit(); }
Alternatively, you can set the stacking directly on an event (e.g., inside a plugin).
var client = exceptionless.ExceptionlessClient.default; // Node.Js // var client = require('exceptionless').ExceptionlessClient.default;
try{ thrownewError('Unable to create order from quote.'); }catch(error){ client.createException(error).setManualStackingKey('MyCustomStackingKey').submit(); }
We're always interested in what you think of Exceptionless' features and functionality, so let us know if you find custom stacking useful, need help implementing it, or just want to chat over on GitHub.
Since the last major release cycle, we've made several minor releases, including Exceptionless 3.2.1, Exceptionless.NET 3.3.5, and Exceptionless.UI 2.3.1.
Lets take a look at some of the highlights, and you can check out the full release notes on each at the provided links, below.
An issue causing query string params and cookies to be renamed when dictionary key names were being changed has been fixed.
client.Register() now respects your session setting.
Manual stacking now uses a model instead of a string, which lets us send a stack title and key value pairs telling the event how to be stacked. More info on the release page.
@ahmettaha added SubmitLog and CreateLog overloads without source parameters. Thanks!
ExceptionlessClient.Default.Configuration.UseDebugLogger() only worked in debug mode with client source, so we replaced it with ExceptionlessClient.Default.Configuration.UseInMemoryLogger()
The serializer wasn't always being passed through so it could get known event data helper methods, which was causing some silent failures - this has been fixed.
@adamzolotarev added the ability to take complete control over how an event is stacked (be careful) by adding the ability to do manual stacking by calling EventBuilder.SetManualStackingKey("MyStackingKey")
You can now ignore events by user agent bot filter when you call (ExceptionlessClient.Default.Configuration.AddUserAgentBotPatterns("*bot*")) on the client side or edit project settings server side.
The default max size limit of RequestInfo has been increased.
Extra nesting has been reduced by merging exception.Data properties into the Error.Data dictionary.
Bug Fix: AggregatedExceptiones that contain more than one inner exception are no longer discarded.
Bug Fix: Machines with a Non-English locale will not process events when SetGeo is used.
Bug Fix: ArgumentNullException will no longer be thrown if post data or cookies contain a null key.
@frankebersoll contributed by adding support for offline storage, which can be enabled by calling client.config.useLocalStorage(). Thanks!
User agent bots can be ignored via a filter now with (`client.config.addUserAgentBotPatterns("bot")) on the client side or via project settings on the server side.
@frankebersoll also added support for manual stacking (be careful! grants complete control). See release notes for instructions.
The implementation of the angular $stateChangeError has also been improved.
There's nothing major to report with the UI, just a few tweaks.
The project settings pages has been reworked by adding the ability to specify user namespaces, and user agents that the clients will ignore. @VikzSharma also fixed an issue where the single page app could be clickjacked - thanks again!
If you've got any questions about any of the release notes above, please don't hesitate to let us know by commenting below or submitting an issue to the respective GitHub repo, above.
That's right! We've re-imagined how session heartbeats and session end events should work on the back end and were able to make them much more efficient, allowing us to stop counting them toward user plan limits!
Exceptionless.NET v3.5.0 and Exceptionless.JavaScript v1.4.0 have been released and they are chock full of new features and feature improvements!
Today we've got a quick article on one of the little known features of Exceptionless: keyboard shortcuts.
Prioritizing your bug fixes and development time in general can be a daunting task.
We've continued to focus some of our attention on our pluggable app foundation block solution, Foundatio, and last week it got some attention from the Microsoft .NET Blog as well!
It's been a while since we introduced filtering and searching when we launched Exceptionless V2.0, so we decided recently that we wanted to take the feedback we've received and do a round of improvements to it.
We’ve learned a lot about simple app deployment since we first started Exceptionless. We initially went with what everyone else was doing (Octopus Deploy), but over time we thought we could greatly simplify and automate it, letting us focus on what matters, improving the product!
Sometimes you just need things to be your way.