1. Prelert Blog

    1. The Secrets to Successful Data Mining

      Explore Anomaly Detection Analytics (Dec 16 2014)

      The Secrets to Successful Data Mining

      Today's IT environments are so complex that IT and DevOps teams cannot reasonably set thresholds and alerts across all meaningful metrics. Even the much-touted best practice of employing standard deviations fails when the actual distribution of values is not Gaussian. Trying to manually define anomalies in increasingly complex systems can result in a deluge of false alerts that waste your IT team's valuable time. In fact, in a random sampling of Splunk users, almost half did not bother setting thresholds for fear of creating additional "alert noise." Scary, huh?

      (Read Full Article)

      Comment Mentions:   Enterprise Management Associates

    2. Why IT Security Teams Need Machine Learning

      Explore Anomaly Detection Analytics (Dec 12 2014)

      Why IT Security Teams Need Machine Learning

      It turns out that finding anomalies in huge volumes of data is exactly what Big Data analytics approaches, such as unsupervised machine learning, are good at...

      (Read Full Article)

      Comment

    3. Security Analytics - Anomaly Detection for IT Security

      Explore Anomaly Detection Analytics (Dec 5 2014)

      Security Analytics - Anomaly Detection for IT Security

      In this short (1:25) video, IT Security professionals protect from advanced threats with security analytics. Machine learning anomaly detection finds the fingerprints of criminial activity in real-time so to detect problems early and act fast...

      (Read Full Article)

      Comment Mentions:   IT Security   Analytics

    4. Data Mining: Don't Settle for Monitoring 1% of Your IT Operations Data

      Explore Anomaly Detection Analytics (Dec 4 2014)

      Data Mining: Don't Settle for Monitoring 1% of Your IT Operations Data

      Do you have the whole automation vs. data mining thing backwards? Traditional IT monitoring approaches automatically analyze less than 1% of the data available looking for 'known bad' behaviors. When a problem is found, an alert is raised that tells us what happened. Troubleshooting teams then have to manually ‘mine’ the other 99% of the data to find out why there was an alarm in the first place. No wonder recent surveys on the state of IT operations verify that two of the biggest concerns are "time spent troubleshooting" and "problems reported by users before IT knows about them."

      (Read Full Article)

      Comment Mentions:   Troubleshooting

    5. Ensure Compliance With IT Operations Analytics

      Explore Anomaly Detection Analytics (Dec 3 2014)

      Ensure Compliance With IT Operations Analytics

      With ITOA (IT Operations Analytics), companies no longer have to remain well-read on each potential flaw or hack that has been concocted and can set up the technology to look for users that are operating outside the definition of normal. The automated nature of ITOA technology frees a company's IT team from having to set thresholds, develop signatures, or just manually search for abnormal behaviors. The ability of ITOA technology to adapt with speed and accuracy minimizes false positives and provides an organization with the resource it needs to weed out any rogue marketers before they get a chance ...

      (Read Full Article)

      Comment Mentions:   Analytics

    6. Implementing StatsReduce in Anomaly Detective

      Explore Anomaly Detection Analytics (Nov 20 2014)

      Implementing StatsReduce in Anomaly Detective

      One of the major additions to version 3.3 of Prelert Anomaly Detective ® for Splunk was a feature called StatsReduce. This feature enables Anomaly Detective to take advantage of Splunk’s distributed processing to analyse immense volumes of data quickly enough to deliver real-time insights. The addition of StatsReduce mode to our Anomaly Detective for Splunk makes it the sole native Splunk app that can deliver real-time analytics for data big enough to require a distributed Splunk installation to store it.

      (Read Full Article)

      Comment

    7. Anomaly Detection on Large Data Sets via Aggregation

      Explore Anomaly Detection Analytics (Nov 18 2014)

      Anomaly Detection on Large Data Sets via Aggregation

      When dealing with very large data sets, there are various practical obstacles, which aren't present at smaller scale, to getting conventional anomaly detection algorithms to work. A key one is the concept of data inertia. This is simply that it is impractical or even impossible to transport the entire data set to a single process. Often, in this context, we have to process a distributed collection of data streams, and we simply don’t have the bandwidth to copy all these data to one process. Therefore, we would like to be able to perform anomaly detection at the level ...

      (Read Full Article)

      Comment

    8. IoT Won’t Work Without Artificial Intelligence

      Explore Anomaly Detection Analytics (Nov 14 2014)

      IoT Won’t Work Without Artificial Intelligence

      As the Internet of Things (IoT) continues its run as one of the most popular technology buzzwords of the year, the discussion has turned from what it is, to how to drive value from it, to the tactical: how to make it work. IoT will produce a treasure trove of big data. This data will hold extremely valuable insight into what’s working well or what’s not – pointing out conflicts that arise and providing high-value insight into new business risks and opportunities as correlations and associations are made. It sounds great. However, the big problem will be finding ways ...

      (Read Full Article)

      Comment Mentions:   Artificial Intelligence

    9. How to Find Anomalies in Splunk's Internal Performance

      Explore Anomaly Detection Analytics (Nov 10 2014)

      How to Find Anomalies in Splunk's Internal Performance

      Splunk does a great job of keeping track of its own internal logs and performance information, and there’s even a very useful and concise app called “S.o.S” (Splunk on Splunk), which tracks and reports on a variety of items culled from Splunk’s “_internal” index and a variety of source logs such as the splunkd.log file. But, just like any visual report or dashboard, there are some fundamental limitations to making this data proactive. Anomaly Detective makes it ridiculously easily to bring machine learning-based anomaly detection to your Splunk on Splunk data!

      (Read Full Article)

      Comment

    10. Prelert Closes $7.5M Investment from Intel Capital and Existing Investors

      Explore Yahoo! Finance (Nov 4 2014)

      Prelert , the anomaly detection company, today announced it has raised a $7.5 million round of venture capital financing from Intel Capital and existing investors, Fairhaven Capital and Sierra Ventures . This investment will enable Prelert to further expand its field sales and engineering organizations to leverage the growing interest in Anomaly Detective® from enterprises, cloud service providers and IT management providers...

      (Read Full Article)

      Comment Mentions:   Intel   CA Technologies   APM

    11. C++11 mutex implementations

      Explore Anomaly Detection Analytics (Nov 3 2014)

      C++11 mutex implementations

      C++11 brought concurrency to standard C++ for the first time. Prior to this the only choice for writing multi-threaded C++ programs was to use a separate C++ library, such as Boost Thread or Intel Thread Building Blocks , or roll your own wrappers around the low-level operating system facilities, such as POSIX threads or Windows threads . Last year I looked into the performance of different types of locks on different platforms. The variation in performance is surprisingly wide. Prelert’s codebase pre-dates C++11, so we have our own wrappers around the low-level operating system facilities. Here are the ones ...

      (Read Full Article)

      Comment Mentions:   Intel

    12. Machine Data is Different (and Why It Matters)

      Explore Anomaly Detection Analytics (Oct 13 2014)

      Machine Data is Different (and Why It Matters)

      Why is machine data different? To answer this question, let’s start by considering different perspectives on what constitutes unstructured data. In the case of a pre-defined set of allowed classifications, the standard approach for benchmarking a machine-learnt classification against a human-generated correct result is to use a confusion matrix . Each of the allowed classifications corresponds to a row and a column in the matrix, and the cells in the matrix record the number of input messages with corresponding human-generated classification and machine-determined classification. The perfect outcome is for all cells of the matrix to contain zeroes except those on ...

      (Read Full Article)

      Comment Mentions:   Prelert

    13. Ready to Talk Anomaly Detection & Advanced Math at Splunk's User Conference

      Explore Anomaly Detection Analytics (Oct 6 2014)

      Ready to Talk Anomaly Detection & Advanced Math at Splunk's User Conference

      The Prelert team, along with partners and customers, will share insights on using machine-based anomaly detection to find value in Big Data in front of over 4,000 IT and business professionals at Splunk’s fifth annual Worldwide Users’ Conference, .conf2014. The event will take place from October 6-9 at the MGM Grand in Las Vegas, Nevada.

      (Read Full Article)

      Comment Mentions:   Prelert   Big Data   Dr. Steve

    14. Anomaly Detection to Reduce the Noise

      Explore Anomaly Detection Analytics (Oct 2 2014)

      Anomaly Detection to Reduce the Noise

      If you have followed some of my other recent blogs, you’ll have noticed that automated anomaly detection is a great technique to find anomalous behaviors in data by effectively contrasting the difference between “normal” and “abnormal. " Most people equate this with contrasting between “good” and “bad,” but that isn’t always necessarily true. What if the data set you’re looking at are “all bad things,” such as Intrusion Detection (IDS) alerts?

      (Read Full Article)

      Comment Mentions:   IT Security

    15. Will You be Replaced by Machine Intelligence?

      Explore Anomaly Detection Analytics (Oct 1 2014)

      Will You be Replaced by Machine Intelligence?

      While humans are definitely needed for the expertise-dependent and creative functions, many aspects of IT operations and performance management could be done more effectively by machine intelligence. Here are just a few examples.

      Deciding What to Monitor
      Most application or service delivery environments have way more metrics, logs and event data than humans can reasonably get their heads around....

      Identifying Normal Behavior
      Even for the 1% of the data we do utilize, it is obvious that setting thresholds and alarm rules is a flawed approach...

      Finding Causal Relationships
      A recent survey by TRAC Research of APM pros turned up the ...

      (Read Full Article)

      Comment Mentions:   Application Performance Management   APM

    16. How to Detect (and Resolve) IT Ops/APM Issues Before Your Users Do

      Explore Anomaly Detection Analytics (Sep 26 2014)

      How to Detect (and Resolve) IT Ops/APM Issues Before Your Users Do

      As originally published by APMdigest. Among the most embarrassing situations for application support teams is first hearing about a critical performance issue from their users. With technology getting increasingly complex and IT environments changing almost overnight, the reality is that even the most experienced support teams are bound to miss a major problem with a critical application or service. One of the contributing factors is their continued reliance on traditional monitoring approaches.

      (Read Full Article)

      Comment Mentions:   APM

    17. Automated Anomaly Detection: A Connector for Amazon CloudWatch

      Explore Anomaly Detection Analytics (Sep 24 2014)

      Automated Anomaly Detection: A Connector for Amazon CloudWatch

      Amazon CloudWatch is a monitoring service for AWS cloud resources and the applications you run on AWS. You can use Amazon CloudWatch to collect and track metrics, collect and monitor log files, and set alarms. At the time of writing, CloudWatch is currently available to all AWS users, with the free version giving basic monitoring metrics (at 5 minute frequency) and generous usage limits. You can also add up to 10 custom metrics and 10 alarms. In this blog I shall explain why it is important to use unsupervised machine learning to effectively manage your AWS environments, and then point ...

      (Read Full Article)

      Comment Mentions:   AWS

    18. Rogue User Detection via Behavioral Analysis

      Explore Anomaly Detection Analytics (Sep 22 2014)

      Rogue User Detection via Behavioral Analysis

      Finding “rogue users” or “rogue systems” using behavioral analysis and automated anomaly detection takes a different approach than the traditional methods of manual data inspection, or the application of rules or signatures to identify specific behavioral violations. A “rogue” user or system, by definition, is someone or something that acts differently from the rest of the population. Therefore, using automated anomaly detection to find behavioral outliers via a comparison of users against each other (peer or behavioral analysis) is a viable approach...

      (Read Full Article)

      Comment Mentions:   IT Security

  1. Recent Articles for IT Ops & APM

    1. 5 Ways to Use APM for Post-Event Security Forensics

      Explore APMdigest (Dec 12 2014)

      Like utilizing video footage from a surveillance camera, captured packets and analysis of network conversations can be retained and looked at retrospectively to detect, clean up and provide detailed information of a breach...

      (Read Full Article)

      Comment Mentions:   Application Performance Management   APM   Breach

    2. Machine Learning Anomaly Detection for IT Ops

      Explore Anomaly Detection Analytics (Dec 10 2014)

      This short (1:20) video demonstrates that IT Ops, APM and DevOps professionals no longer need to use antiquated technology. Machine learning anomaly detection can automate data analysis and detect problems early so that you can act fast - before users detect an issue...

      (Read Full Article)

      Comment Mentions:   APM

    3. Ensure Compliance With IT Operations Analytics

      Explore IT Operations Analytics (ITOA) Landscape (Dec 3 2014)

      With ITOA (IT Operations Analytics), companies no longer have to remain well-read on each potential flaw or hack that has been concocted and can set up the technology to look for users that are operating outside the definition of normal. The automated nature of ITOA technology frees a company's IT team from having to set thresholds, develop signatures, or just manually search for abnormal behaviors. The ability of ITOA technology to adapt with speed and accuracy minimizes false positives and provides an organization with the resource it needs to weed out any rogue marketers before they get a chance ...

      (Read Full Article)

      Comment Mentions:   Kevin Conklin

    4. Network slowdowns are the biggest issue for IT, survey shows

      Explore Computerworld Singapore (Dec 2 2014)

      The study tracked the opinions of more than 400 IT management professionals who work at companies with 1,000 or more employees, and found that the single most common issue faced by the IT pros surveyed recently by Kelton Research is network slowdowns or outages, which outstripped application performance problems, availability and even the dreaded "unanticipated change request."...

      (Read Full Article)

      Comment Mentions:   Application Performance Management

  2. Recent Articles for Security Analytics

    1. Big Data analytics to IT's rescue

      Explore Help Net Security (Dec 11 2014)

      How can an IT security organization ensure they are not the next target (excuse the pun)? It turns out there are common characteristics of successful attacks and that finding anomalies [caused by these breaches ] in huge volumes of data is exactly what Big Data analytics and unsupervised machine learning, are good at.

      (Read Full Article)

      Comment Mentions:   Big Data   Analytics

    2. Prelert to Share Security Expertise and Advice for Preventing Data Breaches at Bloomberg Enterprise Technology Summit

      Explore businesswire.com (Dec 8 2014)

      Stephen Dodson, PhD, Chief Technical Officer, Prelert , will join industry experts at The Bloomberg Enterprise Technology Summit on December 9 at the Vinopolis in London. The event will cover how financial institutions, governments and Fortune 500 companies are leveraging cloud technology, securing the enterprise and addressing emerging regulations, new disruptors and the next generation of start-ups coming out of Silicon Roundabout...

      (Read Full Article)

      Comment Mentions:   Dr. Steve

    3. Cyber Security's Big Data Problem

      Explore eSecurity Planet (Dec 3 2014)

      Big Data has rendered older security models largely obsolete. The all-in-one security product approach that served the industry well some 15 years ago seems hopelessly inadequate today. There is a silver lining to the cloud, however. While the growing use of Big Data has created security issues, Big Data analytics is a part of the solution. The application of real-time analytics in tandem with a methodology that focuses on the threat actors and their likely vectors are the missing ingredients needed to solve ongoing security woes...

      (Read Full Article)

      Comment Mentions:   IBM   Gartner   Big Data

    4. Big Data Security Analytics Landscape

      Explore Work-Bench (Nov 26 2014)

      Big Data Security Analytics is guided by principles rather than a distinct definition, and hence, is somewhat subjective. In fact, many respected analysts, such as Anton Chuvakin at Gartner, don't believe " there is such a market at this time " (note: Gartner does believe that BDSA is relevant as a concept )...

      (Read Full Article)

      Comment Mentions:   IBM   Google   Gartner

  3. Recent Articles for Big Data in IT

    1. The Land of a Thousand Big Data Lakes

      Explore Datanami (Nov 25 2014)

      The prospect of storing and processing all of one’s data in an enterprise data lake running on Hadoop is gaining momentum, particularly when it comes to today’s massive unstructured data flows. However, given what we know of technological evolution and human nature itself, the chance of eliminating data silos and centralizing storage and compute is slim this big-data age...

      (Read Full Article)

      Comment Mentions:   Google   Oracle   Gartner

    2. By turning security into a data problem, we can turn the tables on the bad guys

      Explore t.co / Twitter (Nov 24 2014)

      Security is the science of detecting and preventing attacks against our electronic systems. It's an industry that's swimming in classifiable data, and one that had been crying out for a big data overhaul. Until very recently, security systems were limited to searching in small chunks of that data for a few tell-tale bytes – those being a pattern of data or an antivirus signature unique to a specific attack.Security systems built this way are brittle and resource-intensive. They spend all their time looking for the proverbial byte-sized needle in a data haystack, only to start the whole cycle ...

      (Read Full Article)

      Comment Mentions:   Analytics

    3. Network Security Needs Big Data

      Explore CIO.com (Nov 24 2014)

      According to Gartner , big data analytics will play a crucial role in detecting cyberattacks. By 2016, more than 25% of global organizations will adopt big data analytics for at least one security and fraud-detection use case, up from the current 8%. Big data will change most of the product categories in the field of computer network security, including network monitoring, the authentication and authorization of users, identity management, fraud detection, and systems of governance, risk and compliance. Big data will also change the nature of the security controls, such as conventional firewalls, anti-malware and data loss prevention. In coming years ...

      (Read Full Article)

      Comment Mentions:   IBM   Forrester   Gartner

    4. Cisco hands over security analytics framework to open source development | ZDNet

      Explore Meet New People While Browsing Webpages (Nov 18 2014)

      The OpenSOC framework integrates elements of the Hadoop ecosystem, including Storm, Kafka, and Elasticsearch. According to the firm, this means OpenSOC is capable of full-packet capture indexing, storage, data enrichment, stream processing, batch processing, real-time search, and telemetry aggregation, and also provides a platform that can "effectively enable security analysts to rapidly detect and respond to advanced security threats."

      (Read Full Article)

      Comment Mentions:   Analytics

  4. Recent Articles for Machine Learning Analytics

    1. IoT Won’t Work Without Artificial Intelligence

      Explore Wired.com (Nov 13 2014)

      In an IoT situation, machine learning can help companies take the billions of data points they have and boil them down to what’s really meaningful. The general premise is the same as in the retail applications – review and analyze the data you’ve collected to find patterns or similarities that can be learned from, so that better decisions can be made.

      (Read Full Article)

      Comment Mentions:   Artificial Intelligence   Mark Jaffe

    2. Making Sense of IoT Data With Machine Learning Technologies - Forbes

      Explore forbes.com (Sep 4 2014)

      As companies embark on the long journey of harvesting large amounts of data from connected devices and sensors, the valuable insights hidden in the data are driving up costs and not adding to the bottom line. How can these companies get these insights to market faster while reducing the risk of project failure? One way is to leverage the expertise of companies whose core competency is machine learning. One interesting use case comes from Prelert, a self-described anomaly detection company...

      (Read Full Article)

      Comment Mentions:   Prelert

    3. Sophie Chang Named VP of Engineering at Preler

      Explore businesswire.com (Aug 12 2014)

      Prelert , the anomaly detection company, today announced that it has hired Sophie Chang as Vice President of Engineering to lead its U.K.-based engineering team. In this role, Chang will be responsible for product development and managing all aspects of the team’s activities, helping to enhance Prelert’s machine learning-based anomaly detection engine. Chang brings more than ten years of senior executive experience to her new role, most notably through her time as VP Software at 1E, a fast-growing and successful B2B IT efficiency software company. She was responsible for growing its technology team from two people to ...

      (Read Full Article)

      Comment Mentions:   Prelert   Mark Jaffe   Dr. Steve

    4. How Machine Learning Is Improving Computer Security

      Explore smartdatacollective.com (Jul 27 2014)

      The machine learning approach has a major advantage over the more traditional way of threat detection. With the traditional way, systems had to look for signatures that had already been determined to be a threat. Once these signatures were identified within a network, the system would have to either stop it from further infiltration, or eliminate it. This method has some rather obvious weaknesses, the main one being its non-predictive nature. Machine learning is able to address this major weakness by looking through data for certain patterns and signals, thus predicting future attacks and preventing them, letting the system stay ...

      (Read Full Article)

      Comment

  5. Recent Articles

    1. Juggling Multiple Elasticsearch Instances on a Single Host - Code as Craft

      Explore codeascraft.com (Dec 4 2014)

      Shikhar Bhushan of Etsy writes:

      "Typically, it is assumed that there is a 1:1 relationship between ES instances and machines. This is straightforward and makes sense if your instance requirements line up well with the host – whether physical, virtualized or containerized. We run our clusters on bare metal, and for some of them we have more ES instances than physical hosts. We have good reasons for doing this, and here I’ll share some of the rationale, and the configuration options that we’ve found to be worth tuning..."

      (Read Full Article)

      Comment

    2. Shared Responsibilities for Security in the Cloud, Part 1

      Explore Cloud Security Alliance Blog (Nov 24 2014)

      When businesses owned their applications and all underlying infrastructure, they also owned their security. Now this is changing with a shift in ownership and operational responsibilities over many applications as they are moving to the Cloud. In the cloud era, security is not owned solely by the cloud service provider (CSP) or consumer. Cloud security is a shared responsibility...

      (Read Full Article)

      Comment Mentions:   Cloud Security   Cloud Computing   SaaS

    3. What CIOs Can Learn From the Biggest Data Breaches

      Explore cio.com.au (Nov 12 2014)

      CIO.com tapped several security professional to summarize the origins of the top five recent data breaches to affect U.S. firms. There are also lessons to learn from AT&T , Community Health Systems , Experian , Michaels , Neiman Marcus , P.F. Chang's and the UPS Store , among many others...

      (Read Full Article)

      Comment Mentions:   CIO   Kevin Conklin   CIO.com

    4. Why 2015 is the year of DevOps culture

      Explore Information Age (Nov 11 2014)

      With IT-enabled innovation a competitive differentiator for almost all types and sizes of organisation, agility in delivering IT systems and the ability to run them reliably and cost effectively, is critical. This importance gives rise to the DevOps concept and, in turn, the demand for the skills listed above...

      (Read Full Article)

      Comment

  6. Recent Comments