1. Prelert Blog

    1. Down the Rabbit Hole: How Anomaly Detection Works

      Explore Anomaly Detection Analytics (Apr 15 2014)

      Down the Rabbit Hole: How Anomaly Detection Works

      So what is anomaly detection and how does it work? We worked together with a video crew to explain anomaly detection software in general, as well as our approach to the technical challenges of applying automated anomaly detection on modern data sets. This short (3:29) video is a playful look at how Anomaly Detective acts as your personal machine intelligence partner, learning the normal behavior of your data and monitoring it for abnormal behavior. No more writing rules and thresholds for complex (and often changing) data. Make Anomaly Detective do the work for you so you are no longer ...

      (Read Full Article)


    2. Upgrade Your Security With Population Behavior Analysis

      Explore Anomaly Detection Analytics (Apr 10 2014)

      Upgrade Your Security With Population Behavior Analysis

      Sad but true, if your organization is a prime target for security threats, you should assume that your defenses have already been infiltrated by advanced attackers and it is only a matter of time until the theft or disruption begins. So besides securing your environment, security teams should have another goal, and that is detecting the advanced threats that have successfully penetrated their environment...

      (Read Full Article)

      Comment Mentions:   Breach

    3. 5 Warnings from Your Proxy Server that Could Signal a Hack

      Explore Anomaly Detection Analytics (Apr 8 2014)

      5 Warnings from Your Proxy Server that Could Signal a Hack

      Your organization’s IT perimeter is undoubtedly secured by a complex network consisting of everything from malware detectors and virus scans to vulnerability scanners and intrusion prevention systems. But truly defending your organization requires guarding against advanced threats that have already gotten past your perimeter. One place to start is to heed the alerts generated by your proxy servers, using anomaly detection software to identify behaviors outside the norm. Here are some ways for your IT security team to identify such attacks and shut them down early...

      (Read Full Article)


    4. 7 Ways To Find Rogue Users

      Explore Anomaly Detection Analytics (Apr 3 2014)

      7 Ways To Find Rogue Users

      Amid all the "innocent" acts committed by employees that infringe on corporate policies or accidentally place the company in harm’s way, there are instances of employees who act with the intent to do real harm. Here are 7 steps to finding rogue users...

      (Read Full Article)

      Comment Mentions:   Hadoop   BYOD   IT Security

    5. 3 Ways to Reduce the Amount of Useless Security Alerts

      Explore Anomaly Detection Analytics (Apr 1 2014)

      3 Ways to Reduce the Amount of Useless Security Alerts

      Every day your IT security operations team deals with logs filled with alerts about potential threats to your systems. Most of these alerts are essentially useless, just repeats of minor events that your security systems have seen before and managed with tools already in place. But amid all the “noise” are the fingerprints that identify real threats your IT security team must address. Anomaly detection keeps background chatter in the background, where it belongs, while exposing abnormal behavior patterns...

      (Read Full Article)

      Comment Mentions:   IT Security

    6. Implementing QuickMode for Anomaly Detection

      Explore Anomaly Detection Analytics (Mar 30 2014)

      Implementing QuickMode for Anomaly Detection

      Prelert’s Anomaly Detective for Splunk can operate either in a completely un-intrusive historical diagnostic mode where your data is analysed backwards in time, or in a mode where your data is analysed forwards in time and Prelert persists intermediate state in Splunk summary indexes. We call this second mode of operation Real-Time, although it can also be used with historical data using the new LookBack feature of the app.

      (Read Full Article)


    7. Take the Headaches Out of Web Proxy Data with Anomaly Detection

      Explore Anomaly Detection Analytics (Mar 30 2014)

      Take the Headaches Out of Web Proxy Data with Anomaly Detection

      Corporations around the world have a myriad of perimeter security devices (firewalls, proxy servers, anti-virus, etc. ) in place in order to protect themselves from hackers, malware, and advanced threats. These perimeter devices assist in either restricting access into the enterprise, or restricting access out of the enterprise. In general, an inbound perimeter device (like a firewall) is much more hardened to unauthorized access than an outbound device (like a web proxy).

      (Read Full Article)

      Comment Mentions:   IT Security

    8. 5 Ways To Take Advantage Of IDS And IPS Logs

      Explore Anomaly Detection Analytics (Mar 30 2014)

      5 Ways To Take Advantage Of IDS And IPS Logs

      Once touted as innovative early in their development, Intrusion Detection Systems (IDS) and Intrusion Prevention Systems (IPS) are considered to be a compliance required nuisance to many IT departments because they create volumes of logs so overwhelming as to render them useless. More than just a part of a corporation’s policies, reviews of these logs are often required to ensure your organization remains in compliance.

      (Read Full Article)

      Comment Mentions:   IT Security

    9. How to Identify Abnormal User Behavior with Anomaly Detection

      Explore Anomaly Detection Analytics (Mar 30 2014)

      How to Identify Abnormal User Behavior with Anomaly Detection

      When the topic is Artificial Intelligence (AI), our minds naturally recall the Hollywood imagery of apocalypse that ensues from machines surpassing and then eliminating humans. In reality, AI has manifested itself in our everyday lives in the form of helpful, machine learning applications that accelerate our searches, enrich our entertainment and keep our credit and lifestyle safe.

      (Read Full Article)

      Comment Mentions:   Prelert   Google   Artificial Intelligence

    10. Identifying Potential Data Exfiltrations with Netstat

      Explore Anomaly Detection Analytics (Mar 30 2014)

      Identifying Potential Data Exfiltrations with Netstat

      The last few months have brought heightened focus on the topic of security analytics in general and Advanced Persistant Threats (APTs) in particular. One of the best ways to detect successful APTs is to identify data exfiltrations in process. Automated anomaly detection can be really useful in this regard as the below true customer story illustrates.

      (Read Full Article)

      Comment Mentions:   IT Security   Analytics

    11. The Secret To Finding Unknown IT Security Threats

      Explore Anomaly Detection Analytics (Mar 30 2014)

      The Secret To Finding Unknown IT Security Threats

      There are many technologies your organization can employ to police its perimeter, manage desktop and server configurations, and just deal with daily matters such as known attack patterns, viruses and hacking activity that has occurred in the past. Updating virus software, firewalls and proxy agents are typical efforts that work fine to combat “known” hackers.

      (Read Full Article)

      Comment Mentions:   Big Data   IT Security

    12. q-digest : an algorithm applied to APM for computing approximate quantiles on a collection of integers

      Explore Anomaly Detection Analytics (Mar 30 2014)

      q-digest : an algorithm applied to APM for computing approximate quantiles on a collection of integers

      If you've read our blog, then you'll know that we write software that looks for anomalous behavior in APM data. These can comprise various metrics related to system health, response times, CPU utilization and so on, and also time stamped log messages. All these fall under the umbrella of time series data. The other interesting aspect of APM data is that people are gathering large amounts (terabytes a day is not uncommon) and also want real time or near real time insight into these data as it relates to the health of their system...

      (Read Full Article)

      Comment Mentions:   APM

    13. Anomaly Detection in Unstructured Data

      Explore Prelert (Feb 27 2014)

      Anomaly Detection in Unstructured Data

      When it comes to monitoring, troubleshooting or just analyzing data looking for changes, there are essentially two kinds of data we need to analyze: structured and unstructured. Of the two, “structured” data lends itself to more easy analysis because it consists of well-defined and expected sets of content within each record. In “unstructured” data, however, the content is more freeform – where the meaning of the text within the event is somewhat arbitrary.

      (Read Full Article)

      Comment Mentions:   Rich Collier   Troubleshooting

    14. Anomaly Detection for Security – Overview Video

      Explore Prelert (Feb 25 2014)

      Anomaly Detection for Security – Overview Video

      This recent video of a Google hangout I took part in with editor Keith Shaw of Infoworld has a good overview explanation of the value of Anomaly Detection in security applications...

      (Read Full Article)

      Comment Mentions:   Google

    15. Push Button Machine Learning For Security & IT Ops Analytics

      Explore Prelert (Feb 10 2014)

      Push Button Machine Learning For Security & IT Ops Analytics

      Prelert is applying machine learning to make IT operations and security professionals’ lives easier by automatically alerting them to disconcerting anomalous behaviors of users or resources. Enter a search string, push a button and we automatically learn the patterns in the data and apply the best statistical method to accurately identify anomalies.

      (Read Full Article)

      Comment Mentions:   Prelert   Artificial Intelligence

    16. The Pitfalls of Long Double

      Explore Featured Editorials (Feb 10 2014)

      The Pitfalls of Long Double

      Some might say that using long double improves the accuracy of results. This may be true, but regardless of the amount of digits a fixed precision floating point type has it will be subject to loss of significance if a poorly chosen algorithm is applied to it. Using extended precision rather than double precision may mask this effect in some cases, but in the long term the only solutions are to use algorithms more appropriate for computer calculations or to somehow detect the loss of significance and replace the answer with an appropriate value...

      (Read Full Article)


    17. Get Ready for Machine Learning to Rock Your World

      Explore Prelert (Feb 3 2014)

      Get Ready for Machine Learning to Rock Your World

      Google has just paid a reported $500 million to acquire DeepMind – a London based startup. It sends a clear message that Google knows that in the very near future our world is going to be significantly changed and influenced by a form of artificial intelligence (AI) aptly tagged machine learning. There are now a number of new vendors with applications that combine machine learning with predictive analytics. A classic use case for this technology is anomaly detection software being used by IT organizations to assure the security and performance of the systems that provide on-line services we’ve all come ...

      (Read Full Article)

      Comment Mentions:   Google   Amazon   Big Data

    18. 3 Reasons Users Find Problems before You Do

      Explore Prelert (Jan 28 2014)

      3 Reasons Users Find Problems before You Do

      One of the most embarrassing moments in anybody’s career is the day your boss asks, “how come you didn’t know about that?” And yet survey after survey shows one of the biggest challenges facing IT teams remains end users reporting problems they didn’t know about. Here are 3 big reasons why...

      (Read Full Article)


  1. Recent Articles for IT Ops & APM

    1. Gartner Modifies Stance on APM: Application Aware–Infrastructure Performance Management

      Explore The Virtualization Practice (Apr 2 2014)

      For years, Gartner has insisted that if an APM tool does not cover each of its “Five Dimensions of APM,” one of which is deep code analysis, then it is not an APM tool. Gartner has therefore defined APM to be relevant only to custom-developed applications. Well, it has finally woken up and realized that 70% of the applications that enterprises run are in fact purchased and that maybe the performance of these applications might be important as well. So, Gartner has created a new category, application aware–infrastructure performance management ...

      (Read Full Article)

      Comment Mentions:   Google   Application Performance Management   APM

    2. Elasticsearch Monitoring and Management Plugins

      Explore codecentric Blog (Mar 31 2014)

      Elasticsearch offers a highly useful plugin mechanism as a standard way for extending its core. Plugins enable developers to add new functionality, e.g., a custom analyzer, or provide alternatives to existing functionality, like swapping in another transport module implementation. Additionally, plugins may contain static content which Elasticsearch then serves via its HTTP server...

      (Read Full Article)


    3. Do you know what’s happening in the cloud at your organization?

      Explore Cloud Security Alliance Blog (Mar 26 2014)

      With just a little diligence, you can eliminate the catch-22 of letting people use the cloud apps they want while protecting the enterprise from data loss and network threats. By looking closely at cloud app usage, implementing granular policies, and using data to have a conversation, you can make employees more productive while protecting the business’ interests. Here are ten ways...

      (Read Full Article)

      Comment Mentions:   Cloud Computing   Mission Critical

    4. Cloud Security Monitoring

      Explore The Virtualization Practice (Mar 25 2014)

      There are three implementations of cloud security monitoring: those that modify EUC devices, those that require you go through their gateway sitting within your data center, and those that do neither...

      (Read Full Article)

      Comment Mentions:   Google   Cloud Security   Cloud Computing

  2. Recent Articles for Security Analytics

    1. Feds Look To Big Data On Security Questions

      Explore InformationWeek (Apr 2 2014)

      Government IT leaders believe the growth of big data analytics may provide new tools in combating cyber security threats, according to a new report. The new report -- based on conversations with18 federal government IT leaders with expertise in big data, cybersecurity, and operations -- found that agencies are exploring the opportunities and threats emerging at the intersection of their big data and cybersecurity initiatives....

      (Read Full Article)

      Comment Mentions:   Big Data   Analytics

    2. IT security analytics: the before, during and after

      Explore ComputerWeekly.com (Mar 31 2014)

      A range of security technologies will be required to provide state-of-the-art defenses and there will be no standing still. Those who would steal your data are moving the goalposts all the time and they will be doing that before, during and after their attacks...

      (Read Full Article)

      Comment Mentions:   IBM   EMC   Cisco

    3. CISOs say SIEM not a good choice for big data security analytics

      Explore searchsecurity.techtarget.com (Mar 29 2014)

      Big data security analytics is increasingly a necessity for organizations struggling to spot previously unknown attacks, but according to a trio of CISOs, enterprise IT teams shouldn't plan on using traditional security products such as SIEM for handling large quantities of data...

      (Read Full Article)

      Comment Mentions:   Big Data   Hadoop   Analytics

    4. Big Data, Machine Learning, Behavioral Analytics Combine to Detect & Anticipate Security Threats

      Explore Press Release Distribution (Mar 28 2014)

      Real-time data from every person, application, file and machine can be aggregated and correlated, then mathematically analyzed in a risk equation, applying machine learning to compute and track risks as they are observed, while becoming smarter over time.

      (Read Full Article)

      Comment Mentions:   Big Data   Analytics

  3. Recent Articles for Big Data in IT

    1. Apache Mahout, Hadoop’s original machine learning project, is moving on from MapReduce

      Explore gigaom.com (Mar 27 2014)

      Apache Mahout, a machine learning library for Hadoop since 2009, is joining the exodus away from MapReduce. The project’s community has decided to rework Mahout to support the increasingly popular Apache Spark in-memory data-processing framework, as well as the H2O engine for running machine learning and mathematical workloads at scale...

      (Read Full Article)

      Comment Mentions:   Hadoop

    2. Big Data analytics: The future of IT security?

      Explore techcentral.ie (Mar 24 2014)

      A recent Gartner study highlighted that adoption of big data analytics currently stands at only 8% of large enterprises, though this is set to grow to 25% by 2016 as businesses get to grips with the information being generated across their business...

      (Read Full Article)

      Comment Mentions:   Google   Gartner   Big Data

    3. Record Demand for IT Anomaly Detection in /Security & IT Ops

      Explore Yahoo! Finance (Mar 17 2014)

      With severe national security data breaches during 2013, and a number of national retailers revealing credit card data thefts at the end of the year, IT security was the most popular use case for new anomaly detection. Other popular uses include IT operations, and performance management of critical applications and services Automated anomaly detection and behavioral analytics make it easy for any user or developer to uncover real-time insights into the operational opportunities and risks hidden in massive data sets. IT professionals and app developers who wish to download Prelert Anomaly Detective and begin using it within minutes can do ...

      (Read Full Article)

      Comment Mentions:   Prelert   Mark Jaffe

    4. Splunk partners Symantec to Enhance Security Threat Detection

      Explore arnnet.com.au (Mar 17 2014)

      Symantec has partnered Big Data platform provider, Splunk, to help bolster its security intelligence operations. Symantec will now use Splunk’s enterprise 6 to centralise, monitor and analyse security related data to help investigate incident and detect advanced threats...

      (Read Full Article)

      Comment Mentions:   IBM   Big Data

  4. Recent Articles for Machine Learning Analytics

    1. When Machine Learning Isn’t

      Explore smartdatacollective.com (Apr 12 2014)

      Many startup companies, particularly in the cloud, are touting machine learning capabilities. In some cases, the algorithms are hidden behind a user interface so that users may not know what is happening under the hood. Users may believe that a new capability or algorithm that is closer to artificial intelligence is being used. However, would those same users be excited if they knew that they are buying a very early and immature version of yet another tool to create a decision tree?

      (Read Full Article)

      Comment Mentions:   Analytics

    2. Measure What You Need and No More

      Explore APMdigest (Mar 24 2014)

      Tom Fleck, Senior Software Engineer at OC Systems, said,

      "Projects collect lots of metrics that they do not need. All on this forum would agree that measurement is critical. But not all metrics are useful, and too many metrics can be confusing and obscure what's important."

      And although Tom is on the right track (i.e., humans cannot track all the data that needs to be collected), Tom has neglected to consider the advances of machine learning as applied to vast amounts of IT performance and operational data.  See Anomaly Detective Use Cases.

      (Read Full Article)


    3. Machine learning floats all boats on big data's ocean

      Explore infoworld.com (Mar 7 2014)

      Machine learning's contribution to big data application ROI is twofold: boosting data scientist productivity and uncovering hidden patterns that even the best data scientists may have overlooked.

      (Read Full Article)

      Comment Mentions:   Big Data   Hadoop   Analytics

    4. Using Analytics to Detect Application Performance Anomalies

      Explore APMdigest (Mar 4 2014)

      Instead of alerts based on individual events passing a threshold, the analytical approach is analyzing situations. It compares application behavior against your norms, looking for anomalies that indicate potential problems. Norms are established dynamically using statistical functions such as Bollinger bands, momentum oscillators, standard deviation, velocity, fluctuation and rates of change. This approach ensures that real problems — not just transient variations, a.k.a. "false alarms" — are identified and ensures true readings of real-time performance...

      (Read Full Article)

      Comment Mentions:   Application Performance Management   APM   Big Data

  5. Recent Articles

    1. Make sense of server logs, with logstash, elasticsearch and kibana.

      Explore another LAN (Local Area Network) (Apr 13 2014)

      Logstash, Elasticsearch and Kibana are three different projects that work seamlessly together to create amazing UI dash-boards so you can make sense of dense server logs. Why do yet another “Getting Started with Logstash, Elasticsearch and Kibana” post? I will try to explain why certain steps are important and what you can do to get more out of this setup. I found most of the guides lacking, especially when it comes to kibana dashboards, fixing configuration mistakes, basic elastic search functionality...

      (Read Full Article)


    2. Apache Spark - An Alternative to MapReduce

      Explore Javalobby (Apr 7 2014)

      Apache Spark has started gaining significant momentum and considered to be a promising alternative to support ad-hoc queries and iterative processing logic by replacing MapReduce. It offers interactive code execution using Python and Scala REPL but you can also write and compile your application in Scala and Java. 

      (Read Full Article)

      Comment Mentions:   Big Data   Hadoop   Analytics

    3. Big Data Meets Anomaly Threat Detection

      Explore InformationWeek (Mar 28 2014)

      As cloud adoption and data explosion trends collide, anomaly detection will becomes a critical component of enterprises' security posture and a critical tool for complex problems like advanced persistent threats, data loss, and fraud. Getting it right means proactive control of unknown threats -- which is the holy grail of security...

      (Read Full Article)

      Comment Mentions:   Big Data   Analytics

    4. Will Healthcare Ever Take IT Security Seriously?

      Explore CIO.com (Mar 9 2014)

      A recent threat intelligence study reports widespread security vulnerabilities in healthcare organizations, many of which went unnoticed for months. In December, a developer pulled unencrypted data from a 'certified' mobile health app in less than a minute. Why is it so hard for healthcare to get security right?

      (Read Full Article)

      Comment Mentions:   CIO   IT Security

  6. Recent Comments