1. Prelert Blog

    1. Implementing StatsReduce in Anomaly Detective

      Explore Anomaly Detection Analytics (Nov 20 2014)

      Implementing StatsReduce in Anomaly Detective

      One of the major additions to version 3.3 of Prelert Anomaly Detective ® for Splunk was a feature called StatsReduce. This feature enables Anomaly Detective to take advantage of Splunk’s distributed processing to analyse immense volumes of data quickly enough to deliver real-time insights. The addition of StatsReduce mode to our Anomaly Detective for Splunk makes it the sole native Splunk app that can deliver real-time analytics for data big enough to require a distributed Splunk installation to store it.

      (Read Full Article)

      Comment

    2. Anomaly Detection on Large Data Sets via Aggregation

      Explore Anomaly Detection Analytics (Nov 18 2014)

      Anomaly Detection on Large Data Sets via Aggregation

      When dealing with very large data sets, there are various practical obstacles, which aren't present at smaller scale, to getting conventional anomaly detection algorithms to work. A key one is the concept of data inertia. This is simply that it is impractical or even impossible to transport the entire data set to a single process. Often, in this context, we have to process a distributed collection of data streams, and we simply don’t have the bandwidth to copy all these data to one process. Therefore, we would like to be able to perform anomaly detection at the level ...

      (Read Full Article)

      Comment

    3. IoT Won’t Work Without Artificial Intelligence

      Explore Anomaly Detection Analytics (Nov 14 2014)

      IoT Won’t Work Without Artificial Intelligence

      As the Internet of Things (IoT) continues its run as one of the most popular technology buzzwords of the year, the discussion has turned from what it is, to how to drive value from it, to the tactical: how to make it work. IoT will produce a treasure trove of big data. This data will hold extremely valuable insight into what’s working well or what’s not – pointing out conflicts that arise and providing high-value insight into new business risks and opportunities as correlations and associations are made. It sounds great. However, the big problem will be finding ways ...

      (Read Full Article)

      Comment Mentions:   Artificial Intelligence

    4. How to Find Anomalies in Splunk's Internal Performance

      Explore Anomaly Detection Analytics (Nov 10 2014)

      How to Find Anomalies in Splunk's Internal Performance

      Splunk does a great job of keeping track of its own internal logs and performance information, and there’s even a very useful and concise app called “S.o.S” (Splunk on Splunk), which tracks and reports on a variety of items culled from Splunk’s “_internal” index and a variety of source logs such as the splunkd.log file. But, just like any visual report or dashboard, there are some fundamental limitations to making this data proactive. Anomaly Detective makes it ridiculously easily to bring machine learning-based anomaly detection to your Splunk on Splunk data!

      (Read Full Article)

      Comment

    5. Prelert Closes $7.5M Investment from Intel Capital and Existing Investors

      Explore Yahoo! Finance (Nov 4 2014)

      Prelert , the anomaly detection company, today announced it has raised a $7.5 million round of venture capital financing from Intel Capital and existing investors, Fairhaven Capital and Sierra Ventures . This investment will enable Prelert to further expand its field sales and engineering organizations to leverage the growing interest in Anomaly Detective® from enterprises, cloud service providers and IT management providers...

      (Read Full Article)

      Comment Mentions:   Intel   CA Technologies   APM

    6. C++11 mutex implementations

      Explore Anomaly Detection Analytics (Nov 3 2014)

      C++11 mutex implementations

      C++11 brought concurrency to standard C++ for the first time. Prior to this the only choice for writing multi-threaded C++ programs was to use a separate C++ library, such as Boost Thread or Intel Thread Building Blocks , or roll your own wrappers around the low-level operating system facilities, such as POSIX threads or Windows threads . Last year I looked into the performance of different types of locks on different platforms. The variation in performance is surprisingly wide. Prelert’s codebase pre-dates C++11, so we have our own wrappers around the low-level operating system facilities. Here are the ones ...

      (Read Full Article)

      Comment Mentions:   Intel

    7. Machine Data is Different (and Why It Matters)

      Explore Anomaly Detection Analytics (Oct 13 2014)

      Machine Data is Different (and Why It Matters)

      Why is machine data different? To answer this question, let’s start by considering different perspectives on what constitutes unstructured data. In the case of a pre-defined set of allowed classifications, the standard approach for benchmarking a machine-learnt classification against a human-generated correct result is to use a confusion matrix . Each of the allowed classifications corresponds to a row and a column in the matrix, and the cells in the matrix record the number of input messages with corresponding human-generated classification and machine-determined classification. The perfect outcome is for all cells of the matrix to contain zeroes except those on ...

      (Read Full Article)

      Comment Mentions:   Prelert

    8. Ready to Talk Anomaly Detection & Advanced Math at Splunk's User Conference

      Explore Anomaly Detection Analytics (Oct 6 2014)

      Ready to Talk Anomaly Detection & Advanced Math at Splunk's User Conference

      The Prelert team, along with partners and customers, will share insights on using machine-based anomaly detection to find value in Big Data in front of over 4,000 IT and business professionals at Splunk’s fifth annual Worldwide Users’ Conference, .conf2014. The event will take place from October 6-9 at the MGM Grand in Las Vegas, Nevada.

      (Read Full Article)

      Comment Mentions:   Prelert   Big Data   Dr. Steve

    9. Anomaly Detection to Reduce the Noise

      Explore Anomaly Detection Analytics (Oct 2 2014)

      Anomaly Detection to Reduce the Noise

      If you have followed some of my other recent blogs, you’ll have noticed that automated anomaly detection is a great technique to find anomalous behaviors in data by effectively contrasting the difference between “normal” and “abnormal. " Most people equate this with contrasting between “good” and “bad,” but that isn’t always necessarily true. What if the data set you’re looking at are “all bad things,” such as Intrusion Detection (IDS) alerts?

      (Read Full Article)

      Comment Mentions:   IT Security

    10. Will You be Replaced by Machine Intelligence?

      Explore Anomaly Detection Analytics (Oct 1 2014)

      Will You be Replaced by Machine Intelligence?

      While humans are definitely needed for the expertise-dependent and creative functions, many aspects of IT operations and performance management could be done more effectively by machine intelligence. Here are just a few examples.

      Deciding What to Monitor
      Most application or service delivery environments have way more metrics, logs and event data than humans can reasonably get their heads around....

      Identifying Normal Behavior
      Even for the 1% of the data we do utilize, it is obvious that setting thresholds and alarm rules is a flawed approach...

      Finding Causal Relationships
      A recent survey by TRAC Research of APM pros turned up the ...

      (Read Full Article)

      Comment Mentions:   Application Performance Management   APM

    11. How to Detect (and Resolve) IT Ops/APM Issues Before Your Users Do

      Explore Anomaly Detection Analytics (Sep 26 2014)

      How to Detect (and Resolve) IT Ops/APM Issues Before Your Users Do

      As originally published by APMdigest. Among the most embarrassing situations for application support teams is first hearing about a critical performance issue from their users. With technology getting increasingly complex and IT environments changing almost overnight, the reality is that even the most experienced support teams are bound to miss a major problem with a critical application or service. One of the contributing factors is their continued reliance on traditional monitoring approaches.

      (Read Full Article)

      Comment Mentions:   APM

    12. Automated Anomaly Detection: A Connector for Amazon CloudWatch

      Explore Anomaly Detection Analytics (Sep 24 2014)

      Automated Anomaly Detection: A Connector for Amazon CloudWatch

      Amazon CloudWatch is a monitoring service for AWS cloud resources and the applications you run on AWS. You can use Amazon CloudWatch to collect and track metrics, collect and monitor log files, and set alarms. At the time of writing, CloudWatch is currently available to all AWS users, with the free version giving basic monitoring metrics (at 5 minute frequency) and generous usage limits. You can also add up to 10 custom metrics and 10 alarms. In this blog I shall explain why it is important to use unsupervised machine learning to effectively manage your AWS environments, and then point ...

      (Read Full Article)

      Comment Mentions:   AWS

    13. Rogue User Detection via Behavioral Analysis

      Explore Anomaly Detection Analytics (Sep 22 2014)

      Rogue User Detection via Behavioral Analysis

      Finding “rogue users” or “rogue systems” using behavioral analysis and automated anomaly detection takes a different approach than the traditional methods of manual data inspection, or the application of rules or signatures to identify specific behavioral violations. A “rogue” user or system, by definition, is someone or something that acts differently from the rest of the population. Therefore, using automated anomaly detection to find behavioral outliers via a comparison of users against each other (peer or behavioral analysis) is a viable approach...

      (Read Full Article)

      Comment Mentions:   IT Security

    14. Prelert Takes Home a Silver Stevie Award

      Explore Anomaly Detection Analytics (Sep 15 2014)

      Prelert Takes Home a Silver Stevie Award

      Last Friday marked the twelfth annual American Business Awards and Prelert was honored with a Silver Stevie Award in the New Product or Service of the Year - Software - Big Data Solution category. The announcement was made at the organization’s first ever New Product & Tech Awards banquet at the Palace Hotel in (where else but the tech mecca) San Francisco...

      (Read Full Article)

      Comment Mentions:   Prelert   Big Data

    15. It's Time to Democratize Data Science!

      Explore Anomaly Detection Analytics (Sep 11 2014)

      It's Time to Democratize Data Science!

      Can we realize our full potential by continuously improving advanced analytics that can only be used by data scientists? Is the right answer found in the 2013 prediction from a leading industry analyst that we need to focus our resources on educating millions of data scientists? There is no way that is sustainable. But there is an answer.  Data science can be packaged for the masses – and that is where our focus should be. Want to know how it's done?

      (Read Full Article)

      Comment Mentions:   Intel   Analytics

    16. Why What You Don't Know May Hurt You, & How Security Analytics Can Help

      Explore Anomaly Detection Analytics (Sep 10 2014)

      Why What You Don't Know May Hurt You, & How Security Analytics Can Help

      Attackers try hard to mask their activities and fly below the radar of your security paradigm – but try as they might, in order to accomplish their goals, their behaviors are going to have to be anomalous at some point in time. An authorized login is going to be attempted from a new IP address. A server is going to run a different process than usual. An unusual pattern of data transmissions will occur to a new external URL. The key to mitigating this threat is to be able to identify these ‘fingerprints’ amidst the billions of records produced by the ...

      (Read Full Article)

      Comment Mentions:   IT Security   Analytics

    17. How Security Analytics Help Identify and Manage Breaches

      Explore Anomaly Detection Analytics (Sep 3 2014)

      How Security Analytics Help Identify and Manage Breaches

      Statistical techniques are the only approach that can identify unknown attacks, and even when applied properly will still require a certain amount of human intervention. Security teams can definitely react a lot faster if they are immediately aware of previously unknown threats, so staying ahead of the bad guys really comes down to two things: the speed of a real-time analysis solution and the reaction time of the security team. In the end, this requires that both the right technology and organizational processes are in place...

      (Read Full Article)

      Comment Mentions:   IBM   Dr. Steve   IT Security

    18. Occupy Your Data. Anomaly Detection Stops the Top 1% from Ruling IT.

      Explore Anomaly Detection Analytics (Aug 27 2014)

      Occupy Your Data. Anomaly Detection Stops the Top 1% from Ruling IT.

      How much of your data do you actually pay attention to?  Would you be surprised to realize it is probably far less than 1%?  How about 1% of 1%? This is the case in the vast majority of IT operations, performance management and security shops of any size anywhere in the world. But a typical web app involves hundreds if not thousands of components including software, networks, middleware, app servers, databases, etc. Now consider what happens when something breaks. Most of the time, one of the KPI you've selected triggers an alert or one of the dashboards you ...

      (Read Full Article)

      Comment Mentions:   Application Performance Management

  1. Recent Articles for IT Ops & APM

    1. 18 Tools to Ensure Performance During Cyber Monday and the Holiday Shopping Season

      Explore APMdigest (Nov 13 2014)

      Early detection of potential performance issues is the best way application support teams can ensure a seamless online experience for shoppers on Cyber Monday. "However, everything moves fast and furious during the holiday season, says Kevin Conklin, VP Marketing at Prelert. "So there's not always time to make on-the-fly adjustments to rules or thresholds that flag performance issues. This is where machine learning-powered anomaly detection can be invaluable - without any human intervention, it can identify the early signs of developing problems in massive volumes of data in real-time, enabling IT teams to slash troubleshooting time and decrease the noise ...

      (Read Full Article)

      Comment Mentions:   IBM   BSM   Application Performance Management

    2. Google’s DoubleClick Outage Turns Internet Ad-Free For Over an Hour

      Explore WSJ Blogs (Nov 12 2014)

      A large swath of the Internet ran without advertising for over an hour Wednesday after Google ‘s online ad-serving system DoubleClick for Publishers went down. The outage caused websites run by publishers including BuzzFeed, Time and Forbes to show blank spaces where display ads usually run. Brian O’Kelley, CEO of AppNexus, operator of a large real-time online ad platform and a DoubleClick rival, estimated the disruption cost publishers $1 million per hour in aggregate . Wednesday’s outages affected more than 55,000 websites, according to Dynatrace, which monitors website and web application performance for companies including eight out of ...

      (Read Full Article)

      Comment Mentions:   Google   Gartner   Outage

    3. Application Performance Monitoring, Inside and Out

      Explore EnterpriseNetworkingPlanet (Nov 5 2014)

      Platforms with features designed for easy deployment, such as the inclusion of built-in sensors for common applications, may help enterprises with lean resources stay in the APM game, even as resources move to the cloud. "Having those out of the box, pre-trained and preconfigured, makes the deployment and the selection that much quicker," Jason Lieblich, founder of Exoprise said. Knowing where the organization's blind spots are likely to be once the move to the cloud begins—and taking steps to facilitate visibility into those areas before making the transition—will ensure that the organization is able to maintain performance ...

      (Read Full Article)

      Comment Mentions:   Application Performance Management   Amazon   APM

    4. Apm - at a Crossroad in the Cloud

      Explore APMdigest (Nov 5 2014)

      many organizations, including larger and more sophisticated IT teams, who already have a myriad of systems management and monitoring tools, are seeking alternatives able to help them manage their cloud-based apps. It's this gap – and emerging IT business operations customer base for APM – that will define the APM landscape in the years to come...

      (Read Full Article)

      Comment Mentions:   Google   Microsoft   Application Performance Management

  2. Recent Articles for Security Analytics

    1. Cisco Releases Security Analytics Framework to Open Source

      Explore t.co / Twitter (Nov 18 2014)

      Cisco announced today that it has made available through open source a framework that integrates data analytics tools into security operations. “The OpenSOC framework helps organizations make big data part of their technical security strategy by providing a platform for the application of anomaly detection and incident forensics to the data loss problem,” wrote Pablo Salazar, a Cisco Security Solutions manager in a blog post this morning...

      (Read Full Article)

      Comment Mentions:   Cisco   Analytics

    2. What CIOs Can Learn From the Biggest Data Breaches

      Explore cio.com.au (Nov 12 2014)

      CIO.com tapped several security professional to summarize the origins of the top five recent data breaches to affect U.S. firms. There are also lessons to learn from AT&T , Community Health Systems , Experian , Michaels , Neiman Marcus , P.F. Chang's and the UPS Store , among many others...

      (Read Full Article)

      Comment Mentions:   CIO   Kevin Conklin   CIO.com

    3. Cybersecurity 2014: Breaches and costs rise, confidence and budgets are low

      Explore CSO Online (Nov 5 2014)

      “There are two approaches to figuring out what is happening in your environment. One is threat modeling You determine what your valuable data are to potential adversaries. Determine the ways those adversaries could potentially get to those data, [and then] build a threat model around it” says Mike Rothman, an analyst at the IT security research firm Securosis. The other approach is to baseline enterprise activity. “Then constantly look for anomalous situations that deviate from that baseline." Brian Honan, CEO at Dublin, Ireland-based BH Consulting says, "pretty visualizations and pie-charts don't protect your systems; good actionable information does.”

      (Read Full Article)

      Comment Mentions:   Gartner   Analytics

    4. Prelert Aiming To Make Its Mark In Advanced Security Analytics

      Explore crn.com (Nov 5 2014)

      A new crop of emerging advanced security analytics vendors are promising to exceed security and information event management platforms and provide the visibility and context that incident responders need to investigate the riskiest threats to the network...

      (Read Full Article)

      Comment

  3. Recent Articles for Big Data in IT

    1. Beating Back Cyber Attacks with Big Data + Analytics

      Explore Apache Hadoop Distribution (Oct 28 2014)

      There's a downside to the widespread success of cloud computing, SaaS, smart devices and the plethora of customer and end-user data collected and warehoused in today's business environment: public and private sector organizations are more vulnerable than ever to attacks on their data systems. The more comprehensive, sensitive and greater volume of end user and customer data you warehouse, the more tempting you are to someone wanting to do harm. That said, the same data attracting the threat can be used to thwart an attack. Big data includes all events, activities, actions, and occurrences associated with a threat ...

      (Read Full Article)

      Comment Mentions:   Big Data   Hadoop   SaaS

    2. An Evolution Beyond Security Information & Event Management (SIEM)

      Explore Work-Bench (Oct 28 2014)

      Enterprises now realize that complete prevention of security incidents is impossible. Instead, there must be an increased focus on timely detection and response. Breaches WILL HAPPEN – so find them and contain them quickly. Both classic SIEM and Big Data approaches are compatible with this mindset and seek to unlock value through the aggregation and analysis of events generated by disparate systems. The problem is that SIEM promised the world but under delivered. Verizon’s 2013 Data Breach Investigations Report provides an indication of this, noting that only about 1% of data breaches were discovered through log review. This is due ...

      (Read Full Article)

      Comment Mentions:   Gartner   Big Data   Hadoop

    3. Big Data Security Analytics Landscape

      Explore Work-Bench (Oct 26 2014)

      Big Data Security Analytics is an emerging market and we’re certainly excited to see how it evolves. In the short-term, we’re interested to know which companies we missed. We’ll also be monitoring how well these companies gain traction with enterprise customers. Longer-term, it will be interesting to see which approaches prove most effective at detecting and preventing attackers. 

      (Read Full Article)

      Comment Mentions:   IBM   Gartner   Big Data

    4. Gartner: 8 big trends in big data analytics

      Explore Computerworld (Oct 23 2014)

      "IT managers and implementers cannot use lack of maturity as an excuse to halt experimentation," says Mark Beyer, an analyst at Gartner. Initially, only a few people -- the most skilled analysts and data scientists -- need to experiment. Then those advanced users and IT should jointly determine when to deliver new resources to the rest of the organization. And IT shouldn't necessarily rein in analysts who want to move ahead full-throttle. Rather, Beyer says, IT needs to work with analysts to "put a variable-speed throttle on these new high-powered tools."

      (Read Full Article)

      Comment Mentions:   IBM   Google   Amazon

  4. Recent Articles for Machine Learning Analytics

    1. Making Sense of IoT Data With Machine Learning Technologies - Forbes

      Explore forbes.com (Sep 4 2014)

      As companies embark on the long journey of harvesting large amounts of data from connected devices and sensors, the valuable insights hidden in the data are driving up costs and not adding to the bottom line. How can these companies get these insights to market faster while reducing the risk of project failure? One way is to leverage the expertise of companies whose core competency is machine learning. One interesting use case comes from Prelert, a self-described anomaly detection company...

      (Read Full Article)

      Comment Mentions:   Prelert

    2. Sophie Chang Named VP of Engineering at Preler

      Explore businesswire.com (Aug 12 2014)

      Prelert , the anomaly detection company, today announced that it has hired Sophie Chang as Vice President of Engineering to lead its U.K.-based engineering team. In this role, Chang will be responsible for product development and managing all aspects of the team’s activities, helping to enhance Prelert’s machine learning-based anomaly detection engine. Chang brings more than ten years of senior executive experience to her new role, most notably through her time as VP Software at 1E, a fast-growing and successful B2B IT efficiency software company. She was responsible for growing its technology team from two people to ...

      (Read Full Article)

      Comment Mentions:   Prelert   Mark Jaffe   Dr. Steve

    3. How Machine Learning Is Improving Computer Security

      Explore smartdatacollective.com (Jul 27 2014)

      The machine learning approach has a major advantage over the more traditional way of threat detection. With the traditional way, systems had to look for signatures that had already been determined to be a threat. Once these signatures were identified within a network, the system would have to either stop it from further infiltration, or eliminate it. This method has some rather obvious weaknesses, the main one being its non-predictive nature. Machine learning is able to address this major weakness by looking through data for certain patterns and signals, thus predicting future attacks and preventing them, letting the system stay ...

      (Read Full Article)

      Comment

    4. Big data log analysis thrives on machine learning

      Explore infoworld.com (Jul 7 2014)

      Machine-generated log data is the dark matter of the big data cosmos. It is generated at every layer, node, and component within distributed information technology ecosystems, including smartphones and Internet-of-things endpoints. It is collected, processed, analyzed, and used everywhere, but mostly behind the scenes. Most of it is not designed or intended for direct human analysis. Unless filtered with brutal efficiency, the extreme volumes, velocities, and varieties of log data can quickly overwhelm human cognition. Clearly, automation is key to finding insights within log data, especially as it all scales into big data territory. Automation can ensure that data collection ...

      (Read Full Article)

      Comment

  5. Recent Articles

    1. Why 2015 is the year of DevOps culture

      Explore Information Age (Nov 11 2014)

      With IT-enabled innovation a competitive differentiator for almost all types and sizes of organisation, agility in delivering IT systems and the ability to run them reliably and cost effectively, is critical. This importance gives rise to the DevOps concept and, in turn, the demand for the skills listed above...

      (Read Full Article)

      Comment

    2. Lessons From Searching with PostgreSQL

      Explore The Carbon Emitter (Oct 28 2014)

      We recently built a live-search for over 60,000 records which used both simple pattern matching and full-text search. The records we needed to search were diagnoses that are used to assign patients to a therapy. We had never done full-text search or anything real-time with that many records, so I ended up doing a lot of experimentation. These posts will cover my experience, and I hope they’ll be of value to anyone implementing their own PostgreSQL...

      (Read Full Article)

      Comment

    3. Use ELK To Visualise Security Data

      Explore elasticsearch.org (Oct 21 2014)

      In this blog post, using a virtual machine sitting on the cloud, we’re going to show how to quickly set up a clustered instance of Elasticsearch to visualise firewall and honeypot datasources, namely IPtables and KippoSSH, focusing on the ELK-relevant configuration bits...

      (Read Full Article)

      Comment

    4. Mastering Security Analytics

      Explore Dark Reading (Oct 14 2014)

      When SIEM technology kicked off over a decade ago, the promise was that these platforms would become the catch-all system for storing and correlating security data across the enterprise to help analysts stop attacks in their tracks. But many SIEM platforms still can't pull in all of the necessary feeds to track attacks across the typical attack life cycle, or kill chain, which often spans endpoints, network resources, databases, and so on. Even when they can ingest data from, say, endpoint security systems, they are often unable to normalize it and pair it with related network security data that ...

      (Read Full Article)

      Comment Mentions:   Analytics

  6. Recent Comments