1. Prelert Blog

    1. Prelert Takes Home a Silver Stevie Award

      Explore Anomaly Detection Analytics (Sep 15 2014)

      Prelert Takes Home a Silver Stevie Award

      Last Friday marked the twelfth annual American Business Awards and Prelert was honored with a Silver Stevie Award in the New Product or Service of the Year - Software - Big Data Solution category. The announcement was made at the organization’s first ever New Product & Tech Awards banquet at the Palace Hotel in (where else but the tech mecca) San Francisco...

      (Read Full Article)

      Comment Mentions:   Prelert   Big Data

    2. It's Time to Democratize Data Science!

      Explore Anomaly Detection Analytics (Sep 11 2014)

      It's Time to Democratize Data Science!

      Can we realize our full potential by continuously improving advanced analytics that can only be used by data scientists? Is the right answer found in the 2013 prediction from a leading industry analyst that we need to focus our resources on educating millions of data scientists? There is no way that is sustainable. But there is an answer.  Data science can be packaged for the masses – and that is where our focus should be. Want to know how it's done?

      (Read Full Article)

      Comment Mentions:   Intel   Analytics

    3. Why What You Don't Know May Hurt You, & How Security Analytics Can Help

      Explore Anomaly Detection Analytics (Sep 10 2014)

      Why What You Don't Know May Hurt You, & How Security Analytics Can Help

      Attackers try hard to mask their activities and fly below the radar of your security paradigm – but try as they might, in order to accomplish their goals, their behaviors are going to have to be anomalous at some point in time. An authorized login is going to be attempted from a new IP address. A server is going to run a different process than usual. An unusual pattern of data transmissions will occur to a new external URL. The key to mitigating this threat is to be able to identify these ‘fingerprints’ amidst the billions of records produced by the ...

      (Read Full Article)

      Comment Mentions:   IT Security   Analytics

    4. How Security Analytics Help Identify and Manage Breaches

      Explore Anomaly Detection Analytics (Sep 3 2014)

      How Security Analytics Help Identify and Manage Breaches

      Statistical techniques are the only approach that can identify unknown attacks, and even when applied properly will still require a certain amount of human intervention. Security teams can definitely react a lot faster if they are immediately aware of previously unknown threats, so staying ahead of the bad guys really comes down to two things: the speed of a real-time analysis solution and the reaction time of the security team. In the end, this requires that both the right technology and organizational processes are in place...

      (Read Full Article)

      Comment Mentions:   IBM   Dr. Steve   IT Security

    5. Occupy Your Data. Anomaly Detection Stops the Top 1% from Ruling IT.

      Explore Anomaly Detection Analytics (Aug 27 2014)

      Occupy Your Data. Anomaly Detection Stops the Top 1% from Ruling IT.

      How much of your data do you actually pay attention to?  Would you be surprised to realize it is probably far less than 1%?  How about 1% of 1%? This is the case in the vast majority of IT operations, performance management and security shops of any size anywhere in the world. But a typical web app involves hundreds if not thousands of components including software, networks, middleware, app servers, databases, etc. Now consider what happens when something breaks. Most of the time, one of the KPI you've selected triggers an alert or one of the dashboards you ...

      (Read Full Article)

      Comment Mentions:   Application Performance Management

    6. Data Exfiltration Detection via Behavioral Analysis

      Explore Anomaly Detection Analytics (Aug 21 2014)

      Data Exfiltration Detection via Behavioral Analysis

      There are many possible ways that one can detect “data exfiltration” (data theft), but in many cases, this involves either manual raw data inspection or the application of rules or signatures for specific behavioral violations. An alternative approach is to detect data exfiltration using automated behavioral anomaly detection using data that you’re probably already collecting and storing, and without the use of a DLP-specific security tool.

      (Read Full Article)

      Comment Mentions:   IT Security

    7. The Secret to Fixing Problems Before Users Find Them (part 2)

      Explore Anomaly Detection Analytics (Aug 19 2014)

      The Secret to Fixing Problems Before Users Find Them (part 2)

      In part 1 of this post , we talked about the failed paradigm of using thresholds and rules or 'eyeballs on timecharts' to monitor a critical app or service. In part 2 of this post we'll cover Anomaly Detection products that can leverage data you've already aggregated in stores like Splunk, Elasticsearch or NoSQL databases...

      (Read Full Article)


    8. Choosing bucketSpan Wisely

      Explore Anomaly Detection Analytics (Aug 14 2014)

      Choosing bucketSpan Wisely

      In a previous blog post about optimizing the performance of the Engine API, I mentioned that choosing the proper bucketSpan results in not only a possible performance improvement, but I also alluded to bucketSpan affecting the timeliness and quality of your results. In effect, there is a 3-way balance between performance, timeliness of the results, and quality of the results that I’d like to dig into further here...

      (Read Full Article)


    9. Static code analysis for C++

      Explore Anomaly Detection Analytics (Aug 12 2014)

      Static code analysis for C++

      Static code analysis has long been touted as a must have for high quality software. Unfortunately, my experience with it in previous jobs didn't live up to the hype. Within the last few years the majority of compilers have added a built-in static code analysis capability, so I thought it would be interesting to see how good they are...

      (Read Full Article)


    10. Machine Learning, Anomaly Detection, and the Smart City

      Explore Anomaly Detection Analytics (Aug 7 2014)

      Machine Learning, Anomaly Detection, and the Smart City

      Burdened by heavy traffic, a major metropolitan city wanted to find a solution to help them improve travel times and congestion, and to minimize the effects of incidents and collisions on traffic. Since the city tracks accidents, events, construction, and other road problems, this kind of analysis can be done with automated anomaly detection. By analyzing the traffic and incident data, anomaly detection software can prioritize incidents so that problems with the most impact on traffic are addressed first...

      (Read Full Article)

      Comment Mentions:   Prelert

    11. The Secret to Fixing Problems Before Users Find Them (Part 1)

      Explore Anomaly Detection Analytics (Aug 4 2014)

      The Secret to Fixing Problems Before Users Find Them (Part 1)

      According to a TRAC Research survey on IT performance management challenges the top two issues were 1) 'Problems reported by end-users before IT finds them', and 2) 'too much time spent troubleshooting.' Despite crazy advances in every other field of IT technology, this problem really hasn't changed much in the last 20 years! The good news is we can show you how to change things through the following 3 incremental steps...

      (Read Full Article)

      Comment Mentions:   Google   Application Performance Management   Troubleshooting

    12. Tips to Optimize Performance with the Anomaly Detection Engine API

      Explore Anomaly Detection Analytics (Jul 31 2014)

      Tips to Optimize Performance with the Anomaly Detection Engine API

      As with any piece of software, there are performance considerations. If you’ve followed any of our developer blogs, you’ll quickly realize that Prelert’s engineers take creating high performance software seriously. But, performance is not only in how the software is architected, it is also in how you utilize the software. Here we will discuss some operational techniques that will optimize the performance of the Anomaly Detective Engine API...

      (Read Full Article)

      Comment Mentions:   Prelert

    13. The Unusually Common (Part 1): Methods for Modeling Categorical Data

      Explore Anomaly Detection Analytics (Jul 29 2014)

      The Unusually Common (Part 1): Methods for Modeling Categorical Data

      Recently, in the context of trying to understand how to quantify unusually common categories, I have found myself needing to study various properties of distributions on categorical data. Common building block distributions used to describe categorical data are the Bernoulli and categorical distributions. In fact, the Bernoulli is really just a special case of the categorical distribution with two categories. The categorical distribution is the distribution function of a random variable that takes one of categories m with probabilities { p i } = { p 1 , p 2 , . . . , p m } . The distributions I’m going to focus on are the result of counting ...

      (Read Full Article)


    14. Expanding Data Analysis & Anomaly Detection Using partition & by Fields

      Explore Anomaly Detection Analytics (Jul 24 2014)

      Expanding Data Analysis & Anomaly Detection Using partition & by Fields

      In a previous blog, I showed how easy it is to analyze multiple metrics simultaneously by adding multiple “detectors” to your job configuration definition for the Anomaly Detective Engine API. Now, let’s take it a step further by expanding analysis across instances of things by using “byFieldName” and “partitionFieldName.”

      (Read Full Article)


    15. Java Garbage Collectors: Comparing Performance Options

      Explore Anomaly Detection Analytics (Jul 23 2014)

      Java Garbage Collectors: Comparing Performance Options

      One of the benefits of Java over C++ is that memory management is handled by the JVM, liberating the developer from having to worry about it. At least, that’s the theory. In practice, when you write a non-trivial Java program you do have to consider what’s going on with the memory, but in a different way to writing C++. The JVM uses a garbage collector to find objects that are no longer required and release the memory they occupy. There are many ways that the Java garbage collector can be configured and no single optimum configuration: the most ...

      (Read Full Article)


    16. Proud to be a Finalist: MassTLC Innovative Big Data Technology Award

      Explore Anomaly Detection Analytics (Jul 16 2014)

      Proud to be a Finalist: MassTLC Innovative Big Data Technology Award

      On July 15th at the Microsoft NERD Center , the Mass Technology Leadership Council (MassTLC) announced finalists for its 17th annual Leadership Awards , and we’re proud to let you know that Prelert was named a finalist in the “Innovative Technology of the Year – Big Data” category! With more than 550 member companies, MassTLC is the region’s leading technology association and the premier network for tech executives, entrepreneurs, investors and policy leaders. Prelert is being recognized for its Anomaly Detective software alongside other finalists in the category including EnerNOC, HP Vertica, Pixability and WordStream...

      (Read Full Article)

      Comment Mentions:   Prelert   Big Data

    17. Analyzing Multiple Metrics Using the Anomaly Detection Engine API

      Explore Anomaly Detection Analytics (Jul 15 2014)

      Analyzing Multiple Metrics Using the Anomaly Detection Engine API

      After hearing from some users that they assume it takes multiple jobs to analyze multiple metrics within the Anomaly Detective Engine API, I thought I’d write this blog to explain that analyzing multiple metrics within a single job is a lot easier to accomplish that you might think...

      (Read Full Article)


    18. std::getline is the poor relation

      Explore Anomaly Detection Analytics (Jul 8 2014)

      std::getline is the poor relation

      Instead of using std::getline to read a line at a time from the file, this second program reads chunks of up to 256 bytes from the file, searches for newline characters using the std::find algorithm and then constructs a temporary string that can be moved onto the back of the vector or appends to the string already at the end of the vector if it was incomplete after the previous cycle.  There are all sorts of edge cases that have to be considered.  This is exactly the sort of code that puts people off programming in C++!

      (Read Full Article)


  1. Recent Articles for IT Ops & APM

    1. IT's Continued Challenge to Ensure Performance and Availability

      Explore APMdigest (Sep 8 2014)

      Despite the rise of applications as king of business infrastructure, applications and some form of management tools to support them have been in general business use for decades. However, the struggle for IT to ensure performance and availability of business-critical applications remains...

      (Read Full Article)

      Comment Mentions:   Microsoft   Cisco   Application Performance Management

    2. Making Sense of IoT Data With Machine Learning Technologies - Forbes

      Explore forbes.com (Sep 4 2014)

      As companies embark on the long journey of harvesting large amounts of data from connected devices and sensors, the valuable insights hidden in the data are driving up costs and not adding to the bottom line. How can these companies get these insights to market faster while reducing the risk of project failure? One way is to leverage the expertise of companies whose core competency is machine learning. One interesting use case comes from Prelert, a self-described anomaly detection company...

      (Read Full Article)

      Comment Mentions:   Prelert

    3. Democratizing Machine Learning In Anomaly Detection - Dr. Dobb's

      Explore Dr. Dobb's (Aug 30 2014)

      Written in Python, the Prelert Elasticsearch Connector source is available on GitHub. This enables developers to apply Prelert's machine learning based analytics to fit the big data needs within their own environment...

      (Read Full Article)

      Comment Mentions:   Prelert   Mark Jaffe

    4. Gartner: Traditional Development Practices Will Fail for Mobile Apps

      Explore APMdigest (Aug 20 2014)

      The traditional methods used to define and develop desktop applications will not work with mobile application development (AD), according to Gartner, Inc. Gartner said that as demand from business units in enterprises puts increasing pressure on IT organizations to deliver large numbers of mobile applications, AD teams will have to to employ practices that are different from traditional AD ...
      (Read Full Article)

      Comment Mentions:   IBM   Gartner

  2. Recent Articles for Security Analytics

    1. Finding Unknown Threats with Anomaly Detection

      Explore Information Security News (Aug 25 2014)

      There is a wide array of advanced security technologies that organizations can deploy as perimeter defenses or vulnerability scanners to protect their valuable intellectual property and customer data. Yet, despite their best efforts, attacks and data breaches still happen. Affected companies have to spend a great deal of money to repair the damage to customers (and their reputation). One need look no further than the issues that retail giants like Target had last year to see how critical security is – and how costly it is when it goes wrong...

      (Read Full Article)

      Comment Mentions:   Cisco   Dr. Steve

    2. Understanding and Preventing Cyber Fraud and Cyber Attacks with Advanced Big Data Cyber Security Analytics

      Explore RiskIQ (Aug 22 2014)

      Some security professionals argue that it’s impossible to catch low-impact attacks, because attackers are getting smart about hiding their volume of attacks, essentially trying to fly under the radar and make their attacks look like legitimate traffic. This is often described as the needle in the haystack problem. How can you detect a malicious actor when there’s an overwhelming amount of other pieces of data that don’t mean anything? This is a lie. Data science experts will tell you that no matter how often an abnormal behavior occurs — whether it’s one hundred times or just once ...

      (Read Full Article)

      Comment Mentions:   Big Data   Analytics

    3. How Big Data security analytics is set to transform the security landscape - InformationWeek

      Explore InformationWeek (Aug 22 2014)

      To prevent emerging threats, security tools have to go beyond prevention and piece together different sets of information drawn from different events. For example, today, it is essential for event collection programs to go beyond firewall and IDS events, and add context.  “Identifying anomalous sequences of events at all layers of the stack is not enough. Understanding anomalous activity requires an understanding of the context — the “who, what and why”...

      (Read Full Article)

      Comment Mentions:   IBM   Big Data   Analytics

    4. Big Data Overwhelms Security Teams

      Explore eSecurity Planet (Aug 20 2014)

      A major contributing factor in many recent data breaches has been the fact that many IT security teams are simply overwhelmed by the volume of data they're handling. During last fall's massive Target breach , for example, the company's intrusion detection software triggered several alerts, but Target's security team wasn't able to respond to them.

      According to the results of an ESG survey of 257 enterprise security professionals, 35 percent of respondents said they're challenged by too many false positive alerts and 39 percent are challenged by lack of adequate staffing. Almost one third of ...

      (Read Full Article)

      Comment Mentions:   Dell   Gartner   Big Data

  3. Recent Articles for Big Data in IT

    1. Prelert adds Elasticsearch connector to open up big data

      Explore Betanews (Aug 26 2014)

      Big data analysis can open up valuable insights that are locked up in databases, but releasing that information without access with a team of data scientists isn't easy. Prelert's Elasticsearch Connector is written in Python and is available now via GitHub . Additional connectors for other big data technologies are set to be released in the coming months. Meantime you can find more about anomaly detection on Prelert's website...

      (Read Full Article)

      Comment Mentions:   Prelert   Big Data   Mark Jaffe

    2. When Internet Of Things Meets Big Data

      Explore InformationWeek (Aug 25 2014)

      As always, the burden will fall on IT to solve the data storage, integration, and analytics dilemmas created by the IoT. You can't use your current strategy because the data to be captured, managed, and exploited will be even more diverse, and the use cases even more varied. IT's job will be to identify the best analytics platforms and tools to enable business users to acquire the data they need, analyze its meaning, and act on it quickly. While the range of analytics options available to accommodate big data in general seems broad, the number of systems that ...

      (Read Full Article)

      Comment Mentions:   Big Data   Analytics

    3. The Gartner Hype & Afterwards - #IoT & Big Data | Alton Harewood

      Explore World's Largest Professional Network (Aug 21 2014)

      Gartner suggests that "while interest in big data remains undiminished, it has moved beyond the peak because the market has settled into a reasonable set of approaches, and the new technologies and practices are additive to existing solutions." [Source: Gartner's Hype Cycle Special Report for 2014]...

      (Read Full Article)

      Comment Mentions:   IBM   Google   Microsoft

    4. Gartner Hype Cycle Says: Big Data Out, IoT In

      Explore Datanami (Aug 20 2014)

      As you can see from the graphics below, from 2012 to 2013, big data scaled the “Peak of Inflated Expectations” in Gartner’s Hype Cycle, and in 2014 it went over the edge into the trough. Big data has a few more years before it completely bottoms out in the trough and begins the slow climb up the “Slope of Enlightenment” and ultimately the “Plateau of Productivity.” In other words, we’re still in the era of first-mover advantages...

      (Read Full Article)

      Comment Mentions:   Gartner   Big Data   IDG

  4. Recent Articles for Machine Learning Analytics

    1. Sophie Chang Named VP of Engineering at Preler

      Explore businesswire.com (Aug 12 2014)

      Prelert , the anomaly detection company, today announced that it has hired Sophie Chang as Vice President of Engineering to lead its U.K.-based engineering team. In this role, Chang will be responsible for product development and managing all aspects of the team’s activities, helping to enhance Prelert’s machine learning-based anomaly detection engine. Chang brings more than ten years of senior executive experience to her new role, most notably through her time as VP Software at 1E, a fast-growing and successful B2B IT efficiency software company. She was responsible for growing its technology team from two people to ...

      (Read Full Article)

      Comment Mentions:   Prelert   Mark Jaffe   Dr. Steve

    2. How Machine Learning Is Improving Computer Security

      Explore smartdatacollective.com (Jul 27 2014)

      The machine learning approach has a major advantage over the more traditional way of threat detection. With the traditional way, systems had to look for signatures that had already been determined to be a threat. Once these signatures were identified within a network, the system would have to either stop it from further infiltration, or eliminate it. This method has some rather obvious weaknesses, the main one being its non-predictive nature. Machine learning is able to address this major weakness by looking through data for certain patterns and signals, thus predicting future attacks and preventing them, letting the system stay ...

      (Read Full Article)


    3. Big data log analysis thrives on machine learning

      Explore infoworld.com (Jul 7 2014)

      Machine-generated log data is the dark matter of the big data cosmos. It is generated at every layer, node, and component within distributed information technology ecosystems, including smartphones and Internet-of-things endpoints. It is collected, processed, analyzed, and used everywhere, but mostly behind the scenes. Most of it is not designed or intended for direct human analysis. Unless filtered with brutal efficiency, the extreme volumes, velocities, and varieties of log data can quickly overwhelm human cognition. Clearly, automation is key to finding insights within log data, especially as it all scales into big data territory. Automation can ensure that data collection ...

      (Read Full Article)


    4. Prelert's new anomaly detection platform hopes to replace data scientists

      Explore Tech News (May 22 2014)

      If big data analytic player Prelert has its way, companies will be using its anomaly detection platform instead of turning to data scientists to make sense of it all. “We are putting anomaly detection capabilities into the hands of decision makers,” said Prelert chief executive Mark Jaffe. That is, instead of hiring a team of data scientists to run analyses on your data, Jaffe’s platform will do it for you. And protect it...

      (Read Full Article)

      Comment Mentions:   Prelert   Big Data   Mark Jaffe

  5. Recent Articles

    1. Logstash and Kibana via Docker

      Explore @ehazlett ยท @ehazlett (Aug 31 2014)

      With just a short while playing with the new Kibana and the editor, there is almost a limitless possibility of graphs, etc. for logging. Hopefully this is a quick way to test/deploy it yourself...

      (Read Full Article)


    2. SIEM - The Next Frontier is Security Analytics

      Explore World's Largest Professional Network (Aug 18 2014)

      Security professional are overwhelmed by the sheer volume of notifications and alerts coming from both perimeter security systems and SIEMs; which unfortunately results in these alerts being ignored due to their lack of veracity. Forward thinking and aggressive security analysts are starting to understand through real-world events that in-order to protect their organization, they need to evolve from an all-alert methodology to defining risk patterns that allow what is perceived as a non-disparate security event is linked with sufficient critical asset information allowing a security executive and or security professional to react proactively not reactively...

      (Read Full Article)

      Comment Mentions:   Analytics

    3. Global security analytics market forecast to increase by 10.6% CAGR during 2013 to 2018

      Explore live-pr.com (Aug 18 2014)

      One of the major trends observed in the market is the growing adoption of SaaS-based security solutions. Since SaaS-based security solutions follow the pay-per-use model, they are more affordable than the traditional full license security solutions. In addition, these solutions provide several advantages such as ease of installation and upgradation, leading to their increased adoption...

      (Read Full Article)

      Comment Mentions:   EMC   SaaS   Analytics

    4. Installing Elastic Search on Amazon EC2 instance (Ubuntu 14.04)

      Explore Anoop Raveendran (Aug 17 2014)

      In this article I will explain how to set up an elastic server on EC2 instance. Before going to start we need to have an EC2 instance and I have an instance with Ubuntu 14.10 LTE installed. Normally we setup elastic search server in a separate EC2 instance for isolating the code base from the elastic search server...

      (Read Full Article)

      Comment Mentions:   Amazon   AWS

  6. Recent Comments