Big Data News – 17 Aug 2015

    Top Stories
This week, HP held their Big Data Conference in Boston, bringing domain experts, customers, and HP executives together to discuss trends in big data, analytics, and the new products and announcements from HP.


With the initial buzz surrounding unstructured settling down, the name of the game has become combining the new kinds of information pouring into the enterprise with traditional relational records for integrated analysis. That’s what Hewlett-Packard Co. faciliate to deliver with the latest version of its columnar database that debuted last week.


1010data, the Big Data analytics platform has just been acquired by Advance/Newhouse for $500M. Already providing service to over 750 of the biggest retail and manufacturing companies, the acquisition will provide 1010data the resources to market and grow its integrated cloud data platform offering even further. Currently the platform has analyzed 21 trillion rows of data to deliver actionable insight efficiently. We can only imagine what the future will bring with this new marriage between the two companies.


The term ‘big data’ has become ubiquitous as it has permeated nearly every industry over the last ten years. While definitions abound, its fundamental implications remain constant: it represents a cross-functional focus on leveraging exponentially growing volumes of data to increase operational performance and ROI.


Big data applications are commonplace now. Most large companies have one or more of these applications, which provide fast access to large stores of customer and sales data. As the IT organization grows to install, support and maintain these applications, new job categories and new tasks are added to the mix. These include big data hardware and software support, business analysts who use analytics to probe and explore the data, and managers who must supervise and prioritize job tasks.


I wrote a while ago about the marketing of going public, and how Hortonworks’ early stage IPO was a bold move designed to re-capture focus from the industry. So far the bet seems to be paying off, Hortonworks having recently announced strong numbers. Hortonworks is not alone. There are many IPOs happening or planned for this year. At the same time, we are seeing several large vendors withdraw from the public market. In the integration space alone, TIBCO made the move less than a year ago (September 2014), and Informatica just completed a similar process this month. Looking beyond software, maybe the most visible of such “go private” was Dell, which was taken private by its founder and Silver Lake Capital in 2013.


“This program had absolutely nothing to do with race…but multi-variable equations.” That’s what Brett Goldstein, a former policeman for the Chicago Police Department (CPD) and current Urban Science Fellow at the University of Chicago’s School for Public Policy, said about a predictive policing algorithm he deployed at the CPD in 2010. His algorithm tells police […] The post Big Data Algorithms can Discriminate appeared first on Predictive Analytics Times.


As part of the Internet of Industrial Things (IoIT) research portfolio from the industrial automation and process control practice, this study offers a detailed assessment of key opportunities for Services 2.0 in manufacturing from an application, technology, and market standpoint. Furthermore, the concept of managed services is set to transform the services methodology.


Data scientists are increasingly sought after in the nearshore environment as the needs of SMAC (social, mobile, analytics and cloud) create demand for those who can deal with big data. But what is at the heart of data science and how can it be leveraged in the nearshore?


Parse, Facebook-owned provider of a successful Mobile Backend as a service, has open-sourced its SDK for the iOS/OS X and Android platforms.


Moving to further integrate Hadoop and Vertica, Hewlett-Packard this week released an update to its SQL columnar database that adds support for data streaming and advanced log file text search to better support high-speed analytics typically associated with Internet of Things applications.


Click through for six common myths that should be considered when deploying a BDSA solution, as identified by John Dasher, vice president of marketing at Niara.


Premium marketing services company Harte Hanks HHS recently announced that it is using MapR Technologies Inc.’s MapR Distribution, including Hadoop, to enhance performance, scalability and flexibility of big data solutions. MapR Technologies, the developer and seller of the Apache Hadoop-derived software, is famous for its MapR data platform. This platform has garnered an unparalleled reputation across multiple sectors including large retail, advertisement media, technology and financial services, and hence makes a perfect pick for Harte Hanks.


BM has a big big data play through which it is aiming to be relevant to all market sectors. The company released 20 industry solutions as it targets getting its analytics embedded into all areas of the maturing Big Data market. Andrew Wilcock, VP analytics at IBM UK identified the cross industry nature of the solutions. “Fraud analytics is obviously going to be more applicable to banking, but equally it could also apply to an insurance customer, it could also apply to a banking customer. It’s trying to understand what the business problem is that we are trying to solve.”


Big Data is a major factor driving knowledge discovery and innovation in our information society. However, large amounts of data can only be used efficiently if algorithms for understanding the data are available and if these algorithms can also be appropriately applied in highly scalable systems with thousands of hard drives. Big Data thus presents complex challenges for software developers, as the necessary algorithms can only be created with the aid of specialist skills in a wide range of different fields, such as statistics, machine learning, visualization, databases, and high-performance computing.


This week, HP held their Big Data Conference in Boston, bringing domain experts, customers, and HP executives together to discuss trends in big data, analytics, and the new products and announcements from HP.


After failing with two proof of concept, mainly with scalability issues, when trying to renew their main website the business decided to take a more agile and incremental approach and in a restart of the project inspired by Domain-Driven Design (DDD) having developers talk with domain experts, Chris Patuzzo explains describing the principles of DDD in the context of a real project.


Big data and analytic tools have not yet been harnessed to bring meaningful improvement to the healthcare industry. Use code: PATIMES15 for 15% of a two-day Conference pass to Predictive Analytics World for Healthcare. That’s according to a new report from the National Quality Forum outlining the challenges to making health data and analytics more […] The post Healthcare Challenges: Where Big Data Falls Short appeared first on Predictive Analytics Times.


IOT Design Manifesto 1.0 has been recently released. This manifesto serves as a code of conduct for everyone involved in developing the Internet of Things, outlining 10 principles to help create balanced and honest products in a burgeoning field with many unknowns.


The Art of Scalability is a book on scaling organisations to adapt to web scale growth of their products and services. As well as having technical and architectural implications, scale needs to be dealt with on the organizational level. The goal is to show the reader how to organize technology, people and processes to result in a virtuous circle, a path of continuous improvement to scalability.


Tathagat Varma, shares his experience of working as an individual contributor at a deeper leadership level. He refers to this as an “Individual Leader”. This post explains how to lead without authority. By Savita Pahuja


Christopher Meiklejohn looks at applying two techniques together, deterministic data flow programming and conflict-free replicated data types, to create highly available and fault-tolerant systems.


The growing popularity and use of social media around the world is presenting new opportunities for statisticians to glean insightful information from the infinite stream of posts, tweets and other online communications that will help improve public safety. Two such examples-one that enhances systems to track foodborne illness outbreaks and another designed to improve disaster-response activities-were presented this week at the 2015 Joint Statistical Meetings (JSM 2015) in Seattle. Tracking Foodborne Illness Outbreaks


Mandy Waite shows how to get started with Firebase before walking through a live demo of building a multi-user, collaborative mobile app that provides real-time updates to its users. She also demonstrates how to add user authentication and offline functionality to an application.


“Almost nobody’s competent, Paul. It’s enough to make you cry to see how bad most people are at their jobs. If you can do a half-assed job of anything, you’re a one-eyed man in the kingdom of the blind.” -Kurt Vonnegut, Player Piano Predicting the Unpredictable – a One-Eyed Man in the Kingdom of the […] The post Predictive Modeling the Unpredictable appeared first on Predictive Analytics Times.


Refactoring for Software Design Smells by Girish Suryanarayana, Ganesh Samarthyam, and Tushar Sharma presents a catalogue of typical software design smells and how they can be fixed.


In his role as Co-founder and CEO of Takipi enterprise debugging, Tal Weiss advises enterprises on how to plan and execute production debugging strategies. Tal spoke to InfoQ about best (and worst) tools and practices in this candid interview.


Thomas Schindl presents new tooling features in e(fx)clipse 1.0, the runtime API, components – Code Editor, 3D Viewer-, and tooling API that could be used in other IDE plug-ins.


Ivan Inozemtsev discusses the automated testing difficulties of an Eclipse-based application’s UI, introducing RCP Testing Tool, how it helps and how it is implemented underneath.


Tomorrow marks the anniversary of the VJ-Day and 70 years since the end of World War II. In terms of fatalities, WW2 remains the single deadliest event in all of human history, and the sheer scale of… …


Saul Caganoff looks at what service-oriented enterprises can learn from APIs and microservices to overcome both technical and cultural challenges.


Julien Lavigne du Cadet discusses how Criteo uses Druid: an open-source, real-time data store designed to power interactive applications at scale, covering Druid’s architecture and internals.


Published Date: 2015-08-14 17:18:09 UTC Tags: Big Data, Chief Data Officer, Data Science, Open Data, Technology Title: How Are Higher Education Institutions Using Analytics? Subtitle: How are universities using analytics to help improve their results?


Kevlin Henney discusses unscalable tests, tricks and tips that make tests more specification-like and scalable to large codebases, and choosing between scenario-based and property-based test cases?


Last week, I started a discussion about how customer loyalty is evolving. Today, I’d like to explore how too often, the recognition a customer enjoys as a loyalty member is disconnected from his/her experience elsewhere with the brand and what you can do to change that. Naturally, a customer expects that having a relationship with a brand includes being recognized – after all, she is part of the tribe, right? But what happens when that loyal customer is treated like a stranger, instead of like family? It’s damaging to say the least, and regularly ends the relationship.


This just in from the R Core Team: R 3.2.2 has been released. With this update, data accessed over the Web — including files downloaded using download.file via URLS, and packages installed…


Whether raising a round of funding or creating shareholder wealth, companies increasingly need a well-articulated and demonstrable data and analytics strategy. Here are some things that can sway an investor’s opinion, for good or bad.


Many students at today’s colleges and universities balance their studies with full-time employment. Reflecting on her own daughter’s pressures from attending school and working a full-time job, Jen considers the innovative use of advanced analytics used by some institutions of higher learning to identify students at risk of failure or dropping out.


It was in the third week of July when a startup in the online retail technology space, SmarterHQ, blipped briefly across the industry’s radar. Simon Venture Group, the venture capital arm of the shopping center owner Simon Property Group, had led a financing round for the company, helping it secure $8 million. There was a […] The post The Next Great Online Retail Experience You Have Won’t Be On Amazon appeared first on Predictive Analytics Times.


The past five years have seen the emergence of numerous cybersecurity-based startups but even these Silicon Valley firms can’t solve global information security issues. It will take the power of a large institution to truly turn the tide in the ongoing cyber-battles.


There’s a consensus among retailers: the personalized experience is a must. As researchers at Brand Keys found, brands that can match consumers’ emotional expectations are better able to engage customers and foster loyalty. Armed with smartphones and Internet connections, brands and buyers are enjoying more agency in retail interactions. When looking to buy, customers can research offerings, assess a products’ features, compare prices and read peer reviews. Specialized services to match personal needs are abundant, so each step in the buying process today can be uniquely tailored to the customers’ needs.


Today’s consumer packaged goods (CPG) industry leaders are tackling these pain points head-on with supply chain solutions that utilize data. Thanks to new analytics technologies, organizations have the potential to improve efficiencies, increase speed to reach consumers, build predictive models and provide customization at scale. To get there, supply chain managers have faced some big challenges over the last 10 years.


This eMag focuses on three key areas of “meta-language” for Web APIs: API Description, API Discovery, and API Profiles. You’ll see articles covering all three of these important trends as well as interviews with some of the key personalities in this fast-moving space. By InfoQ


In anticipation of her upcoming conference co-presentation at Predictive Analytics World for Healthcare Boston, Sept 27-Oct 1, 2015, we asked Jessica Taylor, Care Manager at St. Joseph Healthcare, a few questions about incorporating predictive analytics into healthcare. Catch a glimpse of her co-presentation, Improving Care Coordination and Reducing Readmissions Using Real Time Predictive Analytics, and see […] The post Wise Practitioner – Predictive Analytics Interview Series: Jessica Taylor of St. Joseph Healthcare appeared first on Predictive Analytics Times.


Michael Brunton-Spall shows how DevOps-like patterns can be applied on microservices to give the development teams more responsibility for their choices, and how monitoring, logging, auditing, security and other concerns can be managed in a distributed system. By Michael Brunton-Spall


Phil Calcado shares the toolkit and strategy SoundCloud uses to keep its microservices explosion manageable. He presents what they do about the operations overhead, how they spread DevOps skills across teams to support the “you build it, you run it” vision, how they deal with breaking changes and asynchronous behaviors, how they deal with chatty interactions.


Todd Montgomery describes some common problems that have arisen in protocol design, using examples such as HTTP/2, Aeron, etc., and how the solutions can be applied to microservices.


Business both large and small will never be the same again thanks to big data. It continues to revolutionize business processes with more insights and deeper intelligence shared across all business avenues. One of the areas where this data has the most influence is in supply chain management. Use Code PATIMES15 for 15% off a […] The post Big Data is Redefining Supply Chain Management appeared first on Predictive Analytics Times.


In anticipation of his upcoming conference presentation, From Wisdom to Insight: Driving Strategic Decision Making with Predictive Analytics, at Predictive Analytics World for Government, Oct 13-16, 2015, we asked COL William “Bobby” Saxon, Chief, Force Management Enterprise Division, Force Management Directorate, Office of the Deputy Chief of Staff Department of the Army, a few questions […] The post Wise Practitioner – Predictive Analytics Interview Series: COL William Saxon, Department of the Army appeared first on Predictive Analytics Times.


Introduction The Data Scientist’s skill set is a collection of coding and analytic talents, usually emphasizing predictive analytics knowledge over coding acumen. Josh Wills of Cloudera characterizes Data Scientists as people who are, “Better at coding than the average statistician,” which is probably accurate but is also setting the programming bar fairly low :-). Software […] The post Defensive Data Science: What we can Learn from Software Engineers appeared first on Predictive Analytics Times.


In 15 years of experience, I have seen countless predictive models but very few useful ones. Most take too long to build, and then sit unused on the shelf. The model becomes a memory, and a bad one at that. Behind the scenes, countless hours were spent figuring out what to model, getting agreement on definitions […] The post 3 Levers To Success in Predictive Analytics appeared first on Predictive Analytics Times.


Is client loyalty relevant in today’s age?


Speaking Opportunity:  Predictive Analytics World – for Workforce I am honored to again be the Program Chair for Predictive Analytics World / for Workforce.  I am designing the Agenda and an impressive lineup of Speakers for the April 2016 Conference and want to extend a special invitation to our data scientist workforce friends, to speak at this prestigious event. If […] The post Calling All Predictive Workforce Data Scientists… appeared first on Predictive Analytics Times.


In anticipation of her upcoming panel discussion, Insider Threat Panel, at Predictive Analytics World for Government, Oct 13-16, 2015, we asked Patty Larsen, Co-Director, National Insider Threat Task Force, a few questions about her work in predictive analytics. (NOTE: Since the NITTF oversees the Executive Branch’s implementation of insider threat programs, we have unique insight […] The post Wise Practitioner – Predictive Analytics Interview Series: Patty Larsen, Co-Director, National Insider Threat Task Force appeared first on Predictive Analytics Times.


This entry was posted in News. Bookmark the permalink.