1. Prelert Blog

    1. The Secret to Fixing Problems Before Users Find Them (part 2)

      Explore Anomaly Detection Analytics (18 hours, 7 min ago)

      The Secret to Fixing Problems Before Users Find Them (part 2)

      In part 1 of this post , we talked about the failed paradigm of using thresholds and rules or 'eyeballs on timecharts' to monitor a critical app or service. In part 2 of this post we'll cover Anomaly Detection products that can leverage data you've already aggregated in stores like Splunk, Elasticsearch or NoSQL databases...

      (Read Full Article)

      Comment

    2. Choosing bucketSpan Wisely

      Explore Anomaly Detection Analytics (Aug 14 2014)

      Choosing bucketSpan Wisely

      In a previous blog post about optimizing the performance of the Engine API, I mentioned that choosing the proper bucketSpan results in not only a possible performance improvement, but I also alluded to bucketSpan affecting the timeliness and quality of your results. In effect, there is a 3-way balance between performance, timeliness of the results, and quality of the results that I’d like to dig into further here...

      (Read Full Article)

      Comment

    3. Static code analysis for C++

      Explore Anomaly Detection Analytics (Aug 12 2014)

      Static code analysis for C++

      Static code analysis has long been touted as a must have for high quality software. Unfortunately, my experience with it in previous jobs didn't live up to the hype. Within the last few years the majority of compilers have added a built-in static code analysis capability, so I thought it would be interesting to see how good they are...

      (Read Full Article)

      Comment

    4. Machine Learning, Anomaly Detection, and the Smart City

      Explore Anomaly Detection Analytics (Aug 7 2014)

      Machine Learning, Anomaly Detection, and the Smart City

      Burdened by heavy traffic, a major metropolitan city wanted to find a solution to help them improve travel times and congestion, and to minimize the effects of incidents and collisions on traffic. Since the city tracks accidents, events, construction, and other road problems, this kind of analysis can be done with automated anomaly detection. By analyzing the traffic and incident data, anomaly detection software can prioritize incidents so that problems with the most impact on traffic are addressed first...

      (Read Full Article)

      Comment Mentions:   Prelert

    5. The Secret to Fixing Problems Before Users Find Them (Part 1)

      Explore Anomaly Detection Analytics (Aug 4 2014)

      The Secret to Fixing Problems Before Users Find Them (Part 1)

      According to a TRAC Research survey on IT performance management challenges the top two issues were 1) 'Problems reported by end-users before IT finds them', and 2) 'too much time spent troubleshooting.' Despite crazy advances in every other field of IT technology, this problem really hasn't changed much in the last 20 years! The good news is we can show you how to change things through the following 3 incremental steps...

      (Read Full Article)

      Comment Mentions:   Google   Application Performance Management   Troubleshooting

    6. Tips to Optimize Performance with the Anomaly Detection Engine API

      Explore Anomaly Detection Analytics (Jul 31 2014)

      Tips to Optimize Performance with the Anomaly Detection Engine API

      As with any piece of software, there are performance considerations. If you’ve followed any of our developer blogs, you’ll quickly realize that Prelert’s engineers take creating high performance software seriously. But, performance is not only in how the software is architected, it is also in how you utilize the software. Here we will discuss some operational techniques that will optimize the performance of the Anomaly Detective Engine API...

      (Read Full Article)

      Comment Mentions:   Prelert

    7. The Unusually Common (Part 1): Methods for Modeling Categorical Data

      Explore Anomaly Detection Analytics (Jul 29 2014)

      The Unusually Common (Part 1): Methods for Modeling Categorical Data

      Recently, in the context of trying to understand how to quantify unusually common categories, I have found myself needing to study various properties of distributions on categorical data. Common building block distributions used to describe categorical data are the Bernoulli and categorical distributions. In fact, the Bernoulli is really just a special case of the categorical distribution with two categories. The categorical distribution is the distribution function of a random variable that takes one of categories m with probabilities { p i } = { p 1 , p 2 , . . . , p m } . The distributions I’m going to focus on are the result of counting ...

      (Read Full Article)

      Comment

    8. Expanding Data Analysis & Anomaly Detection Using partition & by Fields

      Explore Anomaly Detection Analytics (Jul 24 2014)

      Expanding Data Analysis & Anomaly Detection Using partition & by Fields

      In a previous blog, I showed how easy it is to analyze multiple metrics simultaneously by adding multiple “detectors” to your job configuration definition for the Anomaly Detective Engine API. Now, let’s take it a step further by expanding analysis across instances of things by using “byFieldName” and “partitionFieldName.”

      (Read Full Article)

      Comment

    9. Java Garbage Collectors: Comparing Performance Options

      Explore Anomaly Detection Analytics (Jul 23 2014)

      Java Garbage Collectors: Comparing Performance Options

      One of the benefits of Java over C++ is that memory management is handled by the JVM, liberating the developer from having to worry about it. At least, that’s the theory. In practice, when you write a non-trivial Java program you do have to consider what’s going on with the memory, but in a different way to writing C++. The JVM uses a garbage collector to find objects that are no longer required and release the memory they occupy. There are many ways that the Java garbage collector can be configured and no single optimum configuration: the most ...

      (Read Full Article)

      Comment

    10. Proud to be a Finalist: MassTLC Innovative Big Data Technology Award

      Explore Anomaly Detection Analytics (Jul 16 2014)

      Proud to be a Finalist: MassTLC Innovative Big Data Technology Award

      On July 15th at the Microsoft NERD Center , the Mass Technology Leadership Council (MassTLC) announced finalists for its 17th annual Leadership Awards , and we’re proud to let you know that Prelert was named a finalist in the “Innovative Technology of the Year – Big Data” category! With more than 550 member companies, MassTLC is the region’s leading technology association and the premier network for tech executives, entrepreneurs, investors and policy leaders. Prelert is being recognized for its Anomaly Detective software alongside other finalists in the category including EnerNOC, HP Vertica, Pixability and WordStream...

      (Read Full Article)

      Comment Mentions:   Prelert   Big Data

    11. Analyzing Multiple Metrics Using the Anomaly Detection Engine API

      Explore Anomaly Detection Analytics (Jul 15 2014)

      Analyzing Multiple Metrics Using the Anomaly Detection Engine API

      After hearing from some users that they assume it takes multiple jobs to analyze multiple metrics within the Anomaly Detective Engine API, I thought I’d write this blog to explain that analyzing multiple metrics within a single job is a lot easier to accomplish that you might think...

      (Read Full Article)

      Comment

    12. std::getline is the poor relation

      Explore Anomaly Detection Analytics (Jul 8 2014)

      std::getline is the poor relation

      Instead of using std::getline to read a line at a time from the file, this second program reads chunks of up to 256 bytes from the file, searches for newline characters using the std::find algorithm and then constructs a temporary string that can be moved onto the back of the vector or appends to the string already at the end of the vector if it was incomplete after the previous cycle.  There are all sorts of edge cases that have to be considered.  This is exactly the sort of code that puts people off programming in C++!

      (Read Full Article)

      Comment

    13. Connectors and Results Processors: Anomaly Detection Engine API

      Explore Anomaly Detection Analytics (Jul 3 2014)

      Connectors and Results Processors: Anomaly Detection Engine API

      From time to time, you might hear us reference the terms “Connector” or “Results Processor” in the context of integrating the Anomaly Detective Engine API into a data analysis workflow. So, what roles do each of these components play? The following diagram should help to put this into perspective...

      (Read Full Article)

      Comment Mentions:   Prelert   Hadoop

    14. Good Data for Anomaly Detection

      Explore Anomaly Detection Analytics (Jul 1 2014)

      Good Data for Anomaly Detection

      In general, the best type of data to use with the anomaly detection API engine is time stamped, structured data. This is data that is typically generated from machines (machine data), automated processes, sensor data, monitoring data, performance data, etc. This kind of data spans the disciplines of IT Operations, Application Performance Management, Security Logging/Detection, and more. Generally the data is temporal in nature, and its behavior over time is representative of how well (or not well) a system or a business process is working...

      (Read Full Article)

      Comment Mentions:   Application Performance Management   Big Data

    15. Data Science for the Rest of Us

      Explore Anomaly Detection Analytics (Jun 26 2014)

      Data Science for the Rest of Us

      First and foremost, the goal of Anomaly Detective ® is to bring sophisticated anomaly detection capabilities to the masses. Four kinds of anomalies satisfy a vast majority of use cases in IT Operations, Application Performance Management, Security, and other disciplines. They are: 1) Unexpected changes in the occurrence rate of things, 2) Unexpected changes in the values of things, 3) Rare things, and 4) Population/Peer Outliers. So if you use Splunk, the logical choice is to use Anomaly Detective for Splunk . If have some other data store, and if you're comfortable with the idea of scripting/coding against a ...

      (Read Full Article)

      Comment Mentions:   Prelert   Application Performance Management   Big Data

    16. Discrete Optimization in Machine Learning & Operations Research

      Explore Anomaly Detection Analytics (Jun 24 2014)

      Discrete Optimization in Machine Learning & Operations Research

      This post is going to look at optimization. Optimization is an extremely important problem. Two examples, which might be of particular interest to readers of this blog, are machine learning, which often boils down to optimizing some objective function, plus regularization term, for a set of observed and possibly labeled data, and operations research, which always contains an optimization component.

      (Read Full Article)

      Comment

    17. Detect Developing Issues Before Users With Anomaly Detection

      Explore Anomaly Detection Analytics (Jun 17 2014)

      Detect Developing Issues Before Users With Anomaly Detection

      If you’re still using threshold based alerts in your IT operations, it is very likely that you are only monitoring a small percentage of possible performance indicators for fear of generating even more alert noise. Setting thresholds on key performance indicators (KPIs) is an inexact approach to monitoring for performance issues. The more accuracy you seek, the more labor intensive the process becomes. And in most environments, it is totally impractical to assume you have the resources to accurately alert your team to the vast majority of potential performance issues...

      (Read Full Article)

      Comment Mentions:   APM

    18. 6 Ways To Reduce Troubleshooting Time and SME Involvement

      Explore Anomaly Detection Analytics (Jun 10 2014)

      6 Ways To Reduce Troubleshooting Time and SME Involvement

      When an organization experiences an outage, equipment failure or other serious operations issue, the IT team immediately launches into high gear. Swooping down, the team searches through logs, dashboards and configuration changes in search of the cause. But circling in on the problem often means assembling a team of half dozen or more players, subject matter experts (SMEs) who have to leave their primary responsibilities behind. After hours of following false leads to dead ends, the team finally discovers the root of the problem, and it has nothing to do with half of the people who were called in to ...

      (Read Full Article)

      Comment Mentions:   Troubleshooting   Outage

  1. Recent Articles for IT Ops & APM

    1. Gartner Introduces the Criteria for the 2015 Network Performance Monitoring and Diagnostics (NPMD) Magic Quadrant

      Explore Gartner Blog Network (Aug 15 2014)

      NPMD tools allow for network engineers to understand the performance of applications and infrastructure components via network instrumentation. Additionally, these tools provide insight into the quality of the end user’s experience. The goal of NPMD products is not only to monitor the network components to facilitate outage and degradation resolution, but also to identify performance optimization opportunities. This is conducted via diagnostics, analytics and debugging capabilities to complement additional monitoring of today’s complex IT environments...

      (Read Full Article)

      Comment Mentions:   Gartner   Application Performance Monitoring   SaaS

    2. Sophie Chang Named VP of Engineering at Preler

      Explore businesswire.com (Aug 12 2014)

      Prelert , the anomaly detection company, today announced that it has hired Sophie Chang as Vice President of Engineering to lead its U.K.-based engineering team. In this role, Chang will be responsible for product development and managing all aspects of the team’s activities, helping to enhance Prelert’s machine learning-based anomaly detection engine. Chang brings more than ten years of senior executive experience to her new role, most notably through her time as VP Software at 1E, a fast-growing and successful B2B IT efficiency software company. She was responsible for growing its technology team from two people to ...

      (Read Full Article)

      Comment Mentions:   Prelert   Mark Jaffe   Dr. Steve

    3. Taking IT Operations Beyond Firefighting

      Explore APMdigest (Aug 4 2014)

      Managing highly complex IT environments while trapped in a reactive mode leaves IT managers at a loss for how to understand all causes and effects happening amongst the hundreds of thousands of technologies in use across the enterprise. IT Operations needs to step back and take a more comprehensive approach, breaking the “reactive” cycle...

      (Read Full Article)

      Comment Mentions:   Google   APM   SaaS

    4. The future of Ops, part 1

      Explore Home | SYS-CON MEDIA (Aug 1 2014)

      Marc Andreessen famously stated in 2011 that “software is eating the world”. The world runs on software defined businesses. These businesses realize that in order to be efficient and stay ahead of the competition they must innovate or they will die.

      (Read Full Article)

      Comment Mentions:   Application Performance Management   APM   Application Performance Monitoring

  2. Recent Articles for Security Analytics

    1. Understanding & Preventing Cyber Fraud & Attacks with Advanced Big Data Cyber Security Analytics

      Explore Dr. Chaos (Aug 18 2014)

      Some security professionals argue that it’s impossible to catch low-impact attacks, because attackers are getting smart about hiding their volume of attacks, essentially trying to fly under the radar and make their attacks look like legitimate traffic. This is often described as the needle in the haystack problem. How can you detect a malicious actor when there’s an overwhelming amount of other pieces of data that don’t mean anything? This is a lie. Data science experts will tell you that no matter how often an abnormal behavior occurs — whether it’s one hundred times or just once ...

      (Read Full Article)

      Comment Mentions:   Big Data   Analytics

    2. Global security analytics market forecast to increase by 10.6% CAGR during 2013 to 2018

      Explore live-pr.com (Aug 18 2014)

      One of the major trends observed in the market is the growing adoption of SaaS-based security solutions. Since SaaS-based security solutions follow the pay-per-use model, they are more affordable than the traditional full license security solutions. In addition, these solutions provide several advantages such as ease of installation and upgradation, leading to their increased adoption...

      (Read Full Article)

      Comment Mentions:   EMC   SaaS   Analytics

    3. Predict, Prevent, and Act on Security Threats

      Explore IBM Data magazine (Aug 15 2014)

      Current approaches to network security can combat known threats, but they are not as good at finding new associations or uncovering patterns. As a result, organizations are opening the door to advanced persistent threats (APTs), spear phishing, hacktivism, and other dangers. Within all the noise of big data, organizations need sophisticated real-time analytics to find a relatively weak signal. Without deep insight, most threats cannot be detected. The goal is to predict, prevent, and act on threats to minimize damage, maintain a strong brand image, and keep employees, businesses, and information safe and secure...

      (Read Full Article)

      Comment Mentions:   IBM   InformationWeek   Big Data

    4. Study Finds Security Analytics Significantly Decrease False Alerts

      Explore NewsLocker (Aug 6 2014)

      A study by EMA Research found that ninety percent of organizations who use security analytics have seen a decrease in false alerts or an improvement in actionable alerts by security personnel. Organizations that use security analytics/threat analytics are twice as likely to recover in minutes from unplanned incidents compared to those who don’t use analytics. Organizations who use security analytics/threat analytics are more than 50 percent more likely to have experienced reduced frequency and duration of investigations compared to those who don’t use analytics...

      (Read Full Article)

      Comment Mentions:   Analytics

  3. Recent Articles for Big Data in IT

    1. Big Data Gets a Little More Manageable

      Explore itbusinessedge.com (Aug 15 2014)

      If you’d like to get up to snuff and are primarily interested in open source solutions, I recommend this CIOL.com column by Virenda Gupta , senior vice president at Huawei Technologies India. He discusses new open source solutions in the areas of Big Data processing, analytics and mining. He also addresses Big Data virtualization, where he sees a shortage of comprehensive platforms...

      (Read Full Article)

      Comment Mentions:   Google   Big Data   Hadoop

    2. Big Data A Big Focus Of Security Analytics Products

      Explore Dark Reading (Aug 1 2014)

      There is no shortage of vendors building a case for big data around network forensics and risk management. Here at the RSA Conference, a number of companies -- from IBM to Agiliance to EMC's RSA security division itself -- have made announcements about leveraging big data to improve security...

      (Read Full Article)

      Comment Mentions:   IBM   EMC   Cisco

    3. How security analytics help identify and manage breaches

      Explore Help Net Security (Jul 30 2014)

      It has become a near 'mission impossible' to totally prevent breaches because of the increasingly large and complex environment security professionals are tasked with protecting. We’re even to the point where many organizations already assume they have been successfully breached by advanced persistent attacks, and in this difficult state of affairs, security analytics are extremely important to help us learn everything we can about our environments and the threats they face...

      (Read Full Article)

      Comment Mentions:   IBM   Dr. Steve   Analytics

    4. How Analytics and Big Data are Overcoming the Challenges of APM

      Explore APMdigest (Jul 30 2014)

      The power of big data and data science can help us make the most of the vast cache of APM data we collect and help our DevOps teams supercharge user experience. It’s time to take some of the load off of our humans and let technology make it easier to focus on meaningful changes in user, application and system behavior. Analytics is becoming a valuable component of APM solutions because it’s adding value in so many ways...

      (Read Full Article)

      Comment Mentions:   Application Performance Management   APM   Big Data

  4. Recent Articles for Machine Learning Analytics

    1. How Machine Learning Is Improving Computer Security

      Explore smartdatacollective.com (Jul 27 2014)

      The machine learning approach has a major advantage over the more traditional way of threat detection. With the traditional way, systems had to look for signatures that had already been determined to be a threat. Once these signatures were identified within a network, the system would have to either stop it from further infiltration, or eliminate it. This method has some rather obvious weaknesses, the main one being its non-predictive nature. Machine learning is able to address this major weakness by looking through data for certain patterns and signals, thus predicting future attacks and preventing them, letting the system stay ...

      (Read Full Article)

      Comment

    2. Big data log analysis thrives on machine learning

      Explore infoworld.com (Jul 7 2014)

      Machine-generated log data is the dark matter of the big data cosmos. It is generated at every layer, node, and component within distributed information technology ecosystems, including smartphones and Internet-of-things endpoints. It is collected, processed, analyzed, and used everywhere, but mostly behind the scenes. Most of it is not designed or intended for direct human analysis. Unless filtered with brutal efficiency, the extreme volumes, velocities, and varieties of log data can quickly overwhelm human cognition. Clearly, automation is key to finding insights within log data, especially as it all scales into big data territory. Automation can ensure that data collection ...

      (Read Full Article)

      Comment

    3. Prelert's new anomaly detection platform hopes to replace data scientists

      Explore Tech News (May 22 2014)

      If big data analytic player Prelert has its way, companies will be using its anomaly detection platform instead of turning to data scientists to make sense of it all. “We are putting anomaly detection capabilities into the hands of decision makers,” said Prelert chief executive Mark Jaffe. That is, instead of hiring a team of data scientists to run analyses on your data, Jaffe’s platform will do it for you. And protect it...

      (Read Full Article)

      Comment Mentions:   Prelert   Big Data   Mark Jaffe

    4. Why Machine Learning Matters When Choosing a Big Data Vendor

      Explore smartdatacollective.com (May 14 2014)

      Machine learning is the proper way to make use of all that big data companies are collecting and analyzing. In fact, machine learning is a far more effective way to analyze data since, unlike other methods, it is designed to work with vast amounts of different types of data that is constantly changing. Machine learning has the capability to analyze an entire set of data, not just a small portion of it, allowing for more accurate results. The very nature of machine learning also allows its analytics to operate at a faster pace...

      (Read Full Article)

      Comment Mentions:   Big Data

  5. Recent Articles

    1. Installing Elastic Search on Amazon EC2 instance (Ubuntu 14.04)

      Explore Anoop Raveendran (Aug 17 2014)

      In this article I will explain how to set up an elastic server on EC2 instance. Before going to start we need to have an EC2 instance and I have an instance with Ubuntu 14.10 LTE installed. Normally we setup elastic search server in a separate EC2 instance for isolating the code base from the elastic search server...

      (Read Full Article)

      Comment Mentions:   Amazon   AWS

    2. Installing Logstash on a Windows server with Kibana in IIS

      Explore Ulyaoth (Aug 9 2014)

      This guide shows that it is also possible to run Logstash on a Windows machine and use IIS as web server...

      (Read Full Article)

      Comment

    3. A Roadmap for CIOs & CSOs After the Year of the Mega Breach

      Explore Dark Reading (Aug 1 2014)

      The journey starts with three steps: Engage the C-suite, think like a hacker, and look at the big picture. As high-profile data breaches make headlines, CIOs and CSOs often stand between the C-suite and the next public IT failure. Some may wonder: Is "scapegoat" now a part of IT's job description? Symantec's Internet Security Threat Report shows a 62% increase in the number of data breaches in 2013 from 2012.

      (Read Full Article)

      Comment Mentions:   Cisco   CIO   Analytics

    4. Gartner Says There Is No Security Analytics Market

      Explore Gartner Blog Network (Jul 30 2014)

      Anton Chuvakin, Gartner Research VP wrote:

      "Do we see ANYTHING [that constitutes a market] when “security analytics” is mentioned? No, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no, no!"

      Do you agree?

      (Read Full Article)

      Comment Mentions:   Analytics

  6. Recent Comments