1. Prelert Blog

    1. Proud to be a Finalist: MassTLC Innovative Big Data Technology Award

      Explore Anomaly Detection Analytics (Jul 16 2014)

      Proud to be a Finalist: MassTLC Innovative Big Data Technology Award

      Last night at the Microsoft NERD Center , the Mass Technology Leadership Council (MassTLC) announced finalists for its 17th annual Leadership Awards , and we’re proud to let you know that Prelert was named a finalist in the “Innovative Technology of the Year – Big Data” category! With more than 550 member companies, MassTLC is the region’s leading technology association and the premier network for tech executives, entrepreneurs, investors and policy leaders. Prelert is being recognized for its Anomaly Detective software alongside other finalists in the category including EnerNOC, HP Vertica, Pixability and WordStream...

      (Read Full Article)

      Comment Mentions:   Prelert   Big Data

    2. Analyzing Multiple Metrics Using the Anomaly Detection Engine API

      Explore Anomaly Detection Analytics (Jul 15 2014)

      Analyzing Multiple Metrics Using the Anomaly Detection Engine API

      After hearing from some users that they assume it takes multiple jobs to analyze multiple metrics within the Anomaly Detective Engine API, I thought I’d write this blog to explain that analyzing multiple metrics within a single job is a lot easier to accomplish that you might think...

      (Read Full Article)

      Comment

    3. std::getline is the poor relation

      Explore Anomaly Detection Analytics (Jul 8 2014)

      std::getline is the poor relation

      Instead of using std::getline to read a line at a time from the file, this second program reads chunks of up to 256 bytes from the file, searches for newline characters using the std::find algorithm and then constructs a temporary string that can be moved onto the back of the vector or appends to the string already at the end of the vector if it was incomplete after the previous cycle.  There are all sorts of edge cases that have to be considered.  This is exactly the sort of code that puts people off programming in C++!

      (Read Full Article)

      Comment

    4. Connectors and Results Processors: Anomaly Detection Engine API

      Explore Anomaly Detection Analytics (Jul 3 2014)

      Connectors and Results Processors: Anomaly Detection Engine API

      From time to time, you might hear us reference the terms “Connector” or “Results Processor” in the context of integrating the Anomaly Detective Engine API into a data analysis workflow. So, what roles do each of these components play? The following diagram should help to put this into perspective...

      (Read Full Article)

      Comment Mentions:   Prelert   Hadoop

    5. Good Data for Anomaly Detection

      Explore Anomaly Detection Analytics (Jul 1 2014)

      Good Data for Anomaly Detection

      In general, the best type of data to use with the anomaly detection API engine is time stamped, structured data. This is data that is typically generated from machines (machine data), automated processes, sensor data, monitoring data, performance data, etc. This kind of data spans the disciplines of IT Operations, Application Performance Management, Security Logging/Detection, and more. Generally the data is temporal in nature, and its behavior over time is representative of how well (or not well) a system or a business process is working...

      (Read Full Article)

      Comment Mentions:   Application Performance Management   Big Data

    6. Data Science for the Rest of Us

      Explore Anomaly Detection Analytics (Jun 26 2014)

      Data Science for the Rest of Us

      First and foremost, the goal of Anomaly Detective ® is to bring sophisticated anomaly detection capabilities to the masses. Four kinds of anomalies satisfy a vast majority of use cases in IT Operations, Application Performance Management, Security, and other disciplines. They are: 1) Unexpected changes in the occurrence rate of things, 2) Unexpected changes in the values of things, 3) Rare things, and 4) Population/Peer Outliers. So if you use Splunk, the logical choice is to use Anomaly Detective for Splunk . If have some other data store, and if you're comfortable with the idea of scripting/coding against a ...

      (Read Full Article)

      Comment Mentions:   Prelert   Application Performance Management   Big Data

    7. Discrete Optimization in Machine Learning & Operations Research

      Explore Anomaly Detection Analytics (Jun 24 2014)

      Discrete Optimization in Machine Learning & Operations Research

      This post is going to look at optimization. Optimization is an extremely important problem. Two examples, which might be of particular interest to readers of this blog, are machine learning, which often boils down to optimizing some objective function, plus regularization term, for a set of observed and possibly labeled data, and operations research, which always contains an optimization component.

      (Read Full Article)

      Comment

    8. Detect Developing Issues Before Users With Anomaly Detection

      Explore Anomaly Detection Analytics (Jun 17 2014)

      Detect Developing Issues Before Users With Anomaly Detection

      If you’re still using threshold based alerts in your IT operations, it is very likely that you are only monitoring a small percentage of possible performance indicators for fear of generating even more alert noise. Setting thresholds on key performance indicators (KPIs) is an inexact approach to monitoring for performance issues. The more accuracy you seek, the more labor intensive the process becomes. And in most environments, it is totally impractical to assume you have the resources to accurately alert your team to the vast majority of potential performance issues...

      (Read Full Article)

      Comment Mentions:   APM

    9. 6 Ways To Reduce Troubleshooting Time and SME Involvement

      Explore Anomaly Detection Analytics (Jun 10 2014)

      6 Ways To Reduce Troubleshooting Time and SME Involvement

      When an organization experiences an outage, equipment failure or other serious operations issue, the IT team immediately launches into high gear. Swooping down, the team searches through logs, dashboards and configuration changes in search of the cause. But circling in on the problem often means assembling a team of half dozen or more players, subject matter experts (SMEs) who have to leave their primary responsibilities behind. After hours of following false leads to dead ends, the team finally discovers the root of the problem, and it has nothing to do with half of the people who were called in to ...

      (Read Full Article)

      Comment Mentions:   Troubleshooting   Outage

    10. Open API for Automated Anomaly Detection with Open Source Connectors

      Explore Anomaly Detection Analytics (Jun 5 2014)

      Open API for Automated Anomaly Detection with Open Source Connectors

      Prelert, the leader in big data anomaly detection, recently announced the availability of its anomaly detection engine as an “Open API”.Since many developers and power users already know how to code/script against REST-ful APIs, integrating automated anomaly detection into your environment now becomes extremely easy and you can leave your college statistics textbook in its dusty corner of the attic...

      (Read Full Article)

      Comment Mentions:   Prelert   Big Data   Hadoop

    11. Expert Advice On Reducing Time Spent Triaging IT Issues

      Explore Anomaly Detection Analytics (Jun 3 2014)

      Expert Advice On Reducing Time Spent Triaging IT Issues

      Today’s application environments can easily generate thousands of metrics and log fields that describe performance and behavior. Analyzing this volume of data by visually monitoring dashboards and setting thresholds and alert rules is not a viable option...

      (Read Full Article)

      Comment Mentions:   Oracle

    12. The Secret To Writing Effective IT Rules And Thresholds

      Explore Anomaly Detection Analytics (May 29 2014)

      So what’s the secret to writing IT rules and thresholds? Simply put: Don’t do it! 

      Even those organizations that put considerable effort into thresholds and rules typically monitor only a small fraction of the potentially meaningful data available. In dealing with a few hundred enterprises (large and small) over the past years, our estimation is that less than 1% of the data that could provide immediate value when a performance issues develops is monitored. The typical IT operations environment monitors to know when something is not working and then is left to search through huge volumes of potentially ...

      (Read Full Article)

      Comment Mentions:   Enterprise Management Associates   Troubleshooting

    13. Analyzing only "top N" items

      Explore Anomaly Detection Analytics (May 27 2014)

      A customer of ours recently inquired about the possibility of only analyzing data from his "top 5" clients (he works for a service provider and has many clients, but much of his business comes from his top clients). Normally, one would analyze data using Splunk and Prelert by doing something to the effect of the following...

      (Read Full Article)

      Comment

    14. The Secret To Reducing False Alerts In IT Ops/APM

      Explore Anomaly Detection Analytics (May 22 2014)

      The Secret To Reducing False Alerts In IT Ops/APM

      Your organization’s operations and applications can create a lot of “noise” — alerts generated by your management tools that don’t tell you anything useful. Even when they are accurate in alerting you to a problem that needs your attention, they can be totally lacking in the information you need for troubleshooting. Assembling large teams of subject matter experts (SMEs) to dig through logs and metrics following up on multiple likely causes can chew up a lot of time — one thing you don’t have...

      (Read Full Article)

      Comment Mentions:   APM   Troubleshooting

    15. 6 Benefits Of Anomaly Detection Software For IT Ops/APM

      Explore Anomaly Detection Analytics (May 15 2014)

      6 Benefits Of Anomaly Detection Software For IT Ops/APM

      Dealing with today’s fast-changing world of applications and technology can stretch IT Operations and APM teams pretty thin. Just monitoring your environment to stay ahead of problems will keep a team busy. Add to that the time spent involving subject matter experts in troubleshooting drills and you have the recipe for a lot of frustration. Here are six ways anomaly detection software will give your IT team a break and help to improve your day-to-day IT operations...

      (Read Full Article)

      Comment Mentions:   APM   Troubleshooting

    16. S.o.S. ("Splunk on Splunk")

      Explore Home | Splunk Apps (May 13 2014)

      S.o.S. ("Splunk on Splunk")

      Splunk on Splunk (S.o.S) is an app that turns Splunk's diagnostic tools inward to analyze and troubleshoot problems in your Splunk environment. It contains views and tools that allow you to do the following: View, search and compare Splunk configuration files; Detect and expose errors and anomalies in your installation, including inspection of crash logs; Measure indexing performance and expose event processing bottlenecks; View details of scheduler and user-driven search activity; Analyze data volume metrics captured by Splunk...

      (Read Full Article)

      Comment Mentions:   Troubleshooting   Outage

    17. Anomalies in Splunk-on-Splunk data

      Explore Anomaly Detection Analytics (May 13 2014)

      Anomalies in Splunk-on-Splunk data

      The S.o.S. ("Splunk on Splunk") app is a great example of eating your own dog food - in this case, using Splunk capabilities to monitor and visualize the health and performance of Splunk itself. There is, however, one key challenge - which is that all of the data shown in the S.o.S. app is view only - meaning that one has to be constantly inspecting the S.o.S. dashboards in order to proactively catch problems. However, you can easily augment your S.o.S. data using Prelert Anomaly Detective searches on that data. For example, you can find ...

      (Read Full Article)

      Comment

    18. Why Users Find Problems Before Your IT Monitoring Tools

      Explore Anomaly Detection Analytics (May 8 2014)

      Why Users Find Problems Before Your IT Monitoring Tools

      Among the most embarrassing situations for any IT team is learning about a high profile issue from users and not already knowing about it. But with technology getting increasingly complex and IT environments changing with the pace of business, the reality is that even the most experienced support teams are bound to miss a major problem with a critical application or service. This is especially true if you are relying on traditional monitoring approaches. If your team is ‘eyeballing’ dashboards or dealing with more noisy alerts than you could possibly react to, you will never get ahead of end user ...

      (Read Full Article)

      Comment Mentions:   BYOD

  1. Recent Articles for IT Ops & APM

    1. CA Technologies Seeks Injunction Against AppDynamics'

      Explore NASDAQ Stock Market (Jul 21 2014)

      CA Technologies announced Monday that it has filed counterclaims against AppDynamics, Inc. and its founders in a lawsuit pending in the U.S. District Court for the Northern District of California. It is clear that significant portions of the AppDynamics code was written while its founders were CA employees. CA said it is seeking a court declaration that it owns the code written prior to the departure from CA of AppDynamics' founders as well as any derivative works. CA is also seeking an injunction against AppDynamics prohibiting the misuse of CA's intellectual property and financial damages.

      (Read Full Article)

      Comment Mentions:   CA Technologies   Application Performance Management   APM

    2. Converged networks and application performance management

      Explore Network World (Jul 14 2014)

      These days, the elements in an application delivery environment are so intertwined that it can be difficult to find what’s causing a performance problem without a converged view. It gets more complicated when you add in cloud technologies or third-party services, virtualization, and active optimization technologies.

      (Read Full Article)

      Comment Mentions:   Application Performance Management   Forrester   APM

    3. Understanding Application Performance on the Network – Part II

      Explore apmblog.compuware.com (Jul 11 2014)

      When we think of application performance problems that are network-related, we often immediately think of bandwidth and congestion as likely culprits; faster speeds and less traffic will solve everything, right? This is reminiscent of recent ISP wars; which is better, DSL or cable modems? Cable modem proponents touted the higher bandwidth while DSL proponents warned of the dangers of sharing the network with your potentially bandwidth-hogging neighbors. In this blog entry, we’ll examine these two closely-related constraints...

      (Read Full Article)

      Comment Mentions:   Application Performance Management

    4. Understanding Application Performance on the Network – Part I

      Explore apmblog.compuware.com (Jul 10 2014)

      In an ideal world, your application performance management (APM) solution or your application-aware network performance management (AANPM) solution would automatically isolate the fault domain for you, providing all the diagnostic evidence you need to take the appropriate corrective actions. The reality is that this isn’t always the case; intermittent problems, unexpected application or network behaviors, inefficient configuration settings, or just a desire for more concrete proof mean that manual troubleshooting remains a frequent exercise. And although it may seem like there are a near-unlimited number of root causes of poor application performance, and that trial and error, guesswork and ...

      (Read Full Article)

      Comment Mentions:   Application Performance Management   APM   Troubleshooting

  2. Recent Articles for Security Analytics

    1. Six Ways to Ensure Security for Big Data Analytics

      Explore Home (Jul 16 2014)

      Using Big Data to Secure Big Data Security from big data analytics can be considered in two ways: security derived from big data and security of big data. The big data analytics themselves can provide the information necessary to secure the big data enterprise. Real-time monitoring can be used to support security for big data analytics by using big data itself to identify potential threats to the enterprise before they happen. And then there is securing the big data pools themselves. Security for big data analytics means you have to protect the data...

      (Read Full Article)

      Comment Mentions:   Big Data   Analytics

    2. Big data analytics: the future of IT security?

      Explore Computerworld UK (Jul 15 2014)

      “We are really at the beginning of intelligence-driven security: it is just the tip of the iceberg. Looking forward we are going to have to be smarter [to deal with threats], and we are going to be looking at better data science,” said RSA's head of knowledge delivery and business development, Daniel Cohen...

      (Read Full Article)

      Comment Mentions:   Big Data   IT Security   Analytics

    3. Active Directory Vulnerability Disclosure

      Explore Active Directory Protection (Jul 14 2014)

      As part of our ongoing research on advanced attacks, we expose a critical Active Directory flaw which enables an attacker to change the victim’s password. This attack can be performed despite current identity-theft protection measures. Since 95% of all Fortune 1000 companies have an Active Directory deployment, we consider this vulnerability highly sensitive...

      (Read Full Article)

      Comment Mentions:   Microsoft   Big Data   Analytics

    4. Gartner: Security Practitioners – Stop being a pwnie pawn!

      Explore Gartner Blog Network (Jul 14 2014)

      Lawrence Pingree, Research Director for Gartner, Inc., said:

      "I speak regularly with IT Security organizations that continue to be fearful of their executive management and thus configure relaxed security enforcement policies or implement security controls without any blocking or prevention capabilities turned on. This has got to change if we are to successfully defend and prevent data loss."

      (Read Full Article)

      Comment

  3. Recent Articles for Big Data in IT

    1. Challenges of Big Data Analytics for Security

      Explore infoq.com (Jul 8 2014)

      Although the application of big data analytics to security problems has significant promise, we must address several challenges to realize its true potential. Privacy is particularly relevant as new calls for sharing data among industry sectors and with law enforcement go against the privacy principle of avoiding data reuse—that is, using data only for the purposes that it was collected. Another challenge is the data provenance problem. Because big data lets us expand the data sources we use for processing, it’s hard to be certain that each data source meets the trustworthiness that our analysis algorithms require to ...

      (Read Full Article)

      Comment Mentions:   Cloud Security   Big Data   Hadoop

    2. Understanding big data analytics

      Explore Network World (Jul 7 2014)

      Now that you have initiatives to migrate to the cloud, extend to the mobile web and integrate all the pieces from the bring your own device (BYOD) movement, the next hurdle is going to be keeping an eye on all these moving pieces. IT Operations Analytics (ITOA, what some people call Big Data analytics or Advanced Analytics) has been dubbed the solution to bring order to this rapidly growing complexity. Some are asking “What does this term really mean, what’s the big deal?" Here’s the big deal: the advantage to ITOA is it allows real-time monitoring of huge ...

      (Read Full Article)

      Comment Mentions:   Will Cappelli   Gartner   Cloud Computing

    3. Big data log analysis thrives on machine learning

      Explore infoworld.com (Jul 7 2014)

      Machine-generated log data is the dark matter of the big data cosmos. It is generated at every layer, node, and component within distributed information technology ecosystems, including smartphones and Internet-of-things endpoints. It is collected, processed, analyzed, and used everywhere, but mostly behind the scenes. Most of it is not designed or intended for direct human analysis. Unless filtered with brutal efficiency, the extreme volumes, velocities, and varieties of log data can quickly overwhelm human cognition. Clearly, automation is key to finding insights within log data, especially as it all scales into big data territory. Automation can ensure that data collection ...

      (Read Full Article)

      Comment

    4. The Four IT Security Principles: A Practical Guide to Improving Information Security

      Explore smartdatacollective.com (Jul 7 2014)

      Below are four principles to help you become a more effective IT security leader. While these principles won’t solve all your problems, if you practice them regularly, you can’t help but reduce risks and knock annoying security problems off your to-do lists...

      (Read Full Article)

      Comment Mentions:   CIO   IT Security

  4. Recent Articles for Machine Learning Analytics

    1. Prelert's new anomaly detection platform hopes to replace data scientists

      Explore Tech News (May 22 2014)

      If big data analytic player Prelert has its way, companies will be using its anomaly detection platform instead of turning to data scientists to make sense of it all. “We are putting anomaly detection capabilities into the hands of decision makers,” said Prelert chief executive Mark Jaffe. That is, instead of hiring a team of data scientists to run analyses on your data, Jaffe’s platform will do it for you. And protect it...

      (Read Full Article)

      Comment Mentions:   Prelert   Big Data   Mark Jaffe

    2. Why Machine Learning Matters When Choosing a Big Data Vendor

      Explore smartdatacollective.com (May 14 2014)

      Machine learning is the proper way to make use of all that big data companies are collecting and analyzing. In fact, machine learning is a far more effective way to analyze data since, unlike other methods, it is designed to work with vast amounts of different types of data that is constantly changing. Machine learning has the capability to analyze an entire set of data, not just a small portion of it, allowing for more accurate results. The very nature of machine learning also allows its analytics to operate at a faster pace...

      (Read Full Article)

      Comment Mentions:   Big Data

    3. Machine Learning Matters

      Explore smartdatacollective.com (May 14 2014)

      Machine learning is the proper way to make use of all that big data companies are collecting and analyzing. In fact, machine learning is a far more effective way to analyze data since, unlike other methods, it is designed to work with vast amounts of different types of data that is constantly changing. Machine learning has the capability to analyze an entire set of data, not just a small portion of it, allowing for more accurate results. The very nature of machine learning also allows its analytics to operate at a faster pace...

      (Read Full Article)

      Comment Mentions:   Big Data   Analytics

    4. When Machine Learning Isn’t

      Explore smartdatacollective.com (Apr 12 2014)

      Many startup companies, particularly in the cloud, are touting machine learning capabilities. In some cases, the algorithms are hidden behind a user interface so that users may not know what is happening under the hood. Users may believe that a new capability or algorithm that is closer to artificial intelligence is being used. However, would those same users be excited if they knew that they are buying a very early and immature version of yet another tool to create a decision tree?

      (Read Full Article)

      Comment Mentions:   Analytics

  5. Recent Articles

    1. Java Garbage Collectors: Comparing Performance Options

      Explore Anomaly Detection Analytics (3 hours, 31 min ago)

      One of the benefits of Java over C++ is that memory management is handled by the JVM, liberating the developer from having to worry about it. At least, that’s the theory. In practice, when you write a non-trivial Java program you do have to consider what’s going on with the memory, but in a different way to writing C++. The JVM uses a garbage collector to find objects that are no longer required and release the memory they occupy. There are many ways that the Java garbage collector can be configured and no single optimum configuration: the most ...

      (Read Full Article)

      Comment

    2. How To Use Logstash and Kibana To Centralize Logs On CentOS 6

      Explore digitalocean.com (Jul 14 2014)

      Centralized logging can be very useful when attempting to identify problems with your servers or applications, as it allows you to search through all of your logs in a single place. It is also useful because it allows you to identify issues that span multiple servers by correlating their logs during a specific time frame. This series will get you started with Logstash and Kibana on CentOS 6, then show you how to add more filters to structure your log data...

      (Read Full Article)

      Comment

    3. Root Compromise Vulnerability in instances running ElasticSearch

      Explore VPS Hosting India (Jul 10 2014)

      We are recently observing a spike in root compromise of instances running ElasticSearch and getting affected by the issues explained in the link here. This is a new vulnerability which is not yet documented, and for the moment, we have following recommendations specific to ES which should be reviewed and implemented as soon as possible...

      (Read Full Article)

      Comment

    4. What is Security Analytics and why it is so important?

      Explore Hiding (Jul 9 2014)

      Security Analytics is the application of security intelligence or big data science to full packet capture. It's one of the fastest growing product categories in IT security. It provides a comprehensive view into all network traffic and enables various capabilities, from full packet capture to network forensics and even analysis of long-term historical trends. The primary goal of Security Analytics is to obtain actionable intelligence in real-time, which can be used in doing more advanced threat detection and countering all kind of threats including APTs...

      (Read Full Article)

      Comment Mentions:   Analytics

  6. Recent Comments