We reduced the CPU and Memory load across the entire app. This allows the application to use fewer resources, meaning it has more time to process additional events, making everything faster. Between making Exceptionless and Foundatio 100% Async and these improvements, we've drastically increased the efficiency of the entire platform.
Below, we see the increase in performance from two examples. On the left, we see a reduction in CPU and Memory usage for a deployed web app instance. On the right is a visible reduction in CPU usage for an Elasticsearch node.
Caching efficiency has been improved by removing redundant components that were utilizing valuable resources. For example, we removed the SignalR Redis Backplane, which drastically decreased the number of calls to Redis. Overall, we've made the app smarter throughout regarding how we cache and retrieve data.
We've offloaded long-running API tasks to background jobs, freeing up a lot of resources in the app and allowing us to scale work items independently. For example, marking a stack as fixed or removing a project may take a few moments to be updated now, but the trade-off is worth it. We're working on updating the UI experience to prompt users that the task is running in the background.
Exceptionless is now running on .NET 4.6, which has improved startup time due to various improvements with the new version. Self-hosting users should be sure to upgrade to .NET 4.6 on their servers before updating Exceptionless.
We're always striving to improve the efficiency of Exceptionless and all of our projects. If you see any room for improvement or have any comments when using anything from us, please send us an in-app message, submit a GitHub issue or contact us on the website.
Take a closer look at the new enhancements, below, and head over to the GitHub repo to try Foundatio today. We think you'll find it very handy.
For all our current users, you'll need to upgrade your Foundatio NuGet package and existing Foundatio code to use the async implementations/conventions. The update should be fairly straightforward - we haven't had or heard of any issues so far, including when upgrading Exceptionless.
We've gone 100% async, which means no more blocking synchronous code, allowing your apps to be even more responsive. For more on async and its benefits, take a look at this Microsoft article.
Along with going completely async, we've removed non async methods from the API and simplified the API overall. We wanted to make it easy for developers to roll out new implementations and use the existing API.
We also added new, useful methods across all the APIs to make it easier to perform common tasks. One example includes adding CalculateProgress() to WorkItemHandlerJobs. We hope these helpers allow you to be more productive.
Because we always want more speed out of our apps (and we know you do, too), we used Red Gate's ANTS Performance Profiler to profile various apps, such as Exceptionless, and track down "hot" sections of code. By hot, we mean those pieces of code devoured more resources and took longer to run than all other application code. By doing so, we were able to pinpoint core pieces of Foundatio that were performing poorly compared to what they could be doing. Then we fixed them.
A few optimization examples
Maintenance code in various locations like queues, caches, etc, could and would run in a tight loop, sometimes pegging the CPU to 100% depending on the configuration. By running through and removing that maintenance code, or changing the way it was implemented to be triggered only when needed, rather than running in a constant worker loop, we drastically reduced CPU load and increased efficiency.
We also made massive improvements to queues and locks, on top of the maintenance code changes. Our Dequeue and AcquireLock methods would constantly pull for new items in a loop with a small timeout, which is obviously inefficient. Now we are using async monitors and pulse when a message is received over the message bus. This allows for huge improvements as the code will wait/sleep for a single monitor (of multiple monitors that could be waiting) to get triggered. This means your app isn’t wasting CPU cycles waiting, or external network calls looking for locks or new queue items. Instead, it’s running your app code and getting immediately notified when the lock is available or a new queue item is enqueued. Pretty slick!
Caching got a huge performance boost, too. For InMemoryCache clients, we moved from reflection-based object copying to using Expression Trees. This reduced the time required to get items from cache by a large percentage. Read more about implementing Expression Trees and see the difference it makes here.
We made many other improvements to ensure your apps run fast when using Foundatio, too many to be listed here. If you're already using it, just update your NuGet package to the latest version to take advantage of these improvements. If you're not using it yet, try it out. It's worth a shot, we promise!
We've made it super easy to get up and running, as you'll see below. Take a look and let us know what you think!
Log in to your Exceptionless Dashboard and get your project's API key (Admin > Projects > Select Project > API Keys)
* If you don't have a project, you'll need to create one by going to Admin > Projects > Add New Project
Open the extracted folder on your local machine and go to the "example" folder, then edit index.html.
Replace the API key in the exceptionless.js script tag with your own, and comment out the line with serverURL in it (not needed for demo).
Open index.html in your browser and open your browser's developer tools console so we can see the events fire.
* We've added quick-fire buttons to this page that automatically generate events for you to play around with for the purpose of this demo.
Click on one, generate 100 random events for instance, and you'll see the events queue up in the console. After a few seconds, you should see a queue processing notification telling you it's done processing.
Go back to your Exceptionless project dashboard, and you will see the events hit the system in real time.
* Make sure you are viewing the appropriate project and time period in your dashboard, otherwise you might not see the new events come in.
**Stop** beating your head against the keyboard tracking down those hard-to-find bugs and cut down your development time in general by using Exceptionless to **report and log errors, feature usages, and any other event** within your project.
Learn more below, including how to get started in **just about a minute and a half**!
Now Serving... **5,000,000+ Events Per Month 2000+ Active Projects**
"Exceptionless is awesome. I've STOPPED worrying about whether I'm losing customers to bugs I don't know about." - Eric B.
This month (Aug, '15) we will surpass 5 MILLION EVENTS PER MONTH!
That's pretty huge, so we thought we would celebrate by highlighting a few of the awesome companies that use Exceptionless and throw out a some pretty cool statistics we dug up from the database.
If your company isn't on this list, don't worry - it's not because we don't love you! If you would like your company highlighted on the website, simply send us an in-app message explaining how Exceptionless has helped you squash bugs and improve your development.
User Report is an online survey and feedback tool used to collect information about and from website visitors. The data User Report's tools identifies is useful in usability studies, helps websites better identify their demographics, gives users an easy way to provide feedback, and more. The company's solutions are used by ebay, Toyota, National Geographic, and other large national and international brands.
Used by Facebook, MasterCard, and other large worldwide brands, eMarketer provides insights into the digital marketing world by providing customers with the information they need to run their business effectively and efficiently online. This information includes industry-specific research, insights, and benchmarks.
"Exceptionless allows us to prioritize fixes smarter by showing frequently occurring errors. In real time!" - Ryan Hoffman, eMarketer
Actipro Software provides industry-leading .NET user interface controls for a variety of frameworks, including WPF, WinRT, Silverlight, and WinForms. They are known for their syntax-highlighting code editor control, SyntaxEditor, among other things.
Beth Israel Deaconess Medical Center - Harvard Medical School Teaching Hospital #
BIDMC is one of the nation's preeminent academic medical centers focusing on excellence in clinical care, research, and education.
"The primary reasons that we switched to Exceptionless from our own code is that Exceptionless captures additional detail (including code line numbers), Exceptionless has a better interface and design for reviewing and managing errors and bugs (including summary views that we didn’t previously have), and we no longer need to maintain our own error-logging code." - Philip Dickerson, BIDMC
CodeSmith Tools is a Dallas-based software development company focused on creating software tools that increase developer productivity. Their flagship product, CodeSmith Generator, has seen huge success over the years as a template driven source code generator that automates common application source code creation for text-based languages. They also provide a variety of frameworks for generating proven architectures.
IKA is a laboratory equipment maker in the research and development field that builds innovative magnetic stirrers, mixers, overhead stirrers, shakers, homogenizers, mills, rotary evaporators, calorimeters, laboratory reactors and the specially developed software for analysis applications.
"I’m using Exceptionless to report bugs /crashes which I encounter routinely while testing the UI. It helps me zero in on what could be the issue, AND have a documented track of what all errors I’ve seen... I’d say that Exceptionless helps me find all the bugs which I’d have to spend more time on zeroing in on, faster." - Bharat Mallapur, IKA
Once again, we'd like to extend our thanks to all the awesome users we have that make Exceptionless such a great project to work on and use. Everyone has provided excellent feedback, allowing us to continue pushing towards a faster, more efficient, more effective solution for all your error and event reporting needs.
Getting your self hosted instance of Exceptionless up and running is super easy. In the video below, Blake does the entire install on his local machine and is sending events to Exceptionless in less than 2 minutes!
Exceptionless has come a long way, and it's only going to get easier, faster, and more robust moving forward. Our users are a huge part of that, so please submit an issue on GitHub or send us an in-app message if you have any feedbacks, bugs, installation questions, etc. Thanks!
Previously, MongoDB was a major dependency that increase the complexity of the overall project. All the data previously hosted in MongoDB is now hosted in ElasticSearch, making it super easy for users to self host or develop Exceptionless since you only need to set up ElasticSearch (which you had to do anyway).
With this, Redis is no longer configured by default, but you can set it up easily by setting the connection string. We definitely recommend using it.
Removal of the MongoDB dependency brings us one step closer to running Exceptionless on vNext, on any operating system. We hope to achieve this soon, but do not have a timeline.
The goal is to make self hosting as easy as possible so anyone can set it up and try Exceptionless out.
With Exceptionless 3.0, we now have a single build artifact that contains bo the SPA app and the API end server, with default configuration. The ZIP file contains a batch file you can run to download and start ElasticSearch, launch IIS Express with a temp website, and load your browser automatically with the Exceptionless test instance. This lets you load up everything and play around with Exceptionless in a single click!
Another change to the configuration is that you now have the ability to set every Exceptionless API Setting via Environmental Variables.
In order to make it easier and faster to deploy, we removed the OctopusDeploy build dependency and are now using Azure Git deploy, which pulls directly from a GitHub repository that contains the build artifacts.
With this move, our mindset changed regarding the master branch. For us it means production, but for you it means that whatever is in our master branch is stable and currently deployed to live. We no longer have to wait for a build to complete, create a production release in OctopusDeploy, then manually deploy it, we just commit to git and the rest is history!
Expect more on this topic from us soon. In the mean time, enjoy the simplified build process.
The only users that need to worry about upgrading anything for this new release are self hosters. If you are self hosting Exceptionless, please review the Self Hosting Documentation, which contains information about upgrading your existing install.
In the process of developing Exceptionless, we realized there was a lack of good, simple, open source solutions for caching, queues, locks, messaging, jobs, file storage, and metrics when building scaleable applications.
We tried an open source Redis cache client for caching, but it went commercial (expensive) and there wasn't any in-memory implementations. For the message bus, we looked at NServiceBus (good product, high cost, not open source friendly) and MassTransit (local setup a pain, Azure support lacking), but were left disappointed. For storage, we simply could not find a solution that was decoupled and supported in memory, file storage, or Azure Blog Storage.
So, naturally, we built our own!
Meet Foundatio - your key to painless, scalable development and testing for your app! Let's take a look at some examples, below.
Below are small examples of what is possible with Foundatio caching, queues, locks, messaging, jobs, file storage, and metrics. We hope you find these explanations and samples useful, but please let us know if you have any questions or comments.
Foundatio provides four cache implementations, all derived from the ICacheClient interface, that save you expensive operations when creating or getting data by allowing you to store and access the data super fast.
For Exceptionless, we use RedisHybridCacheClient to cache users, organizations, and projects, which has a huge performance boost since we don't have to serialize the item if it's in local memory cache.
Run jobs by calling Run() or passing it to the JobRunner class, which allows you to easily run jobs as Azure Web Jobs. All jobs are required to derive from the JobBase class.
Around here, we use the jobs feature for processing events, sending mail messages, and more. Some jobs, such as our DailySummaryJob, which we only want to run once, also use locks (above) to only run one instance.
Metrics are used throughout the Exceptionless system to provide insight into how the system is working and get external alerts if events are not processing or report how much load is on the current system.
As with any development project, Foundatio is a work in progress and we will continue developing it, however it is extremely stable and used in production by various companies, including Exceptionless (obviously). The next major steps are full Async support (some implementations already have it), and vnext support.
Naturally, we want to know what you think and what we should work on next, so please let us know!