Big Data News – 9 Mar 2016

Today's Infographic Link: How Music Affects Your Driving

Top Stories
Career website Glassdoor today released its 2016 report on the "Highest Paying Jobs In Demand." In the list of the top 25 U.S. jobs, 11 were in tech — more than any other industry (health care and finance were next in line, with three jobs each). Glassdoor's list is ordered by average base salary (not to be confused with the top jobs list). For a job title to be considered, it had to receive at least 75 salary reports shared by U.S.-based employees over the past year. Without further ado, here are the U.S. tech jobs that made the cut: Software Development Manager Software Architect IT Manager Solutions Architect Applications Development Manager Systems Architect Data Scientist Data Architect Product Manager Analytics Manager Information Systems Manager The first four tech jobs placed in the top 10 of the full list. Software development manager was No. 1 in tech, and it placed fourth overall.

People with epilepsy suffer from recurrent, unprovoked seizures that can cause injury and even death from "sudden unexpected death in epilepsy" (SUDEP), a condition that occurs minutes after a seizure ends. Now Empatica, co-founded by MIT professor and wearables pioneer Rosalind Picard, has developed a medical-quality consumer wristband, called Embrace, that monitors stress signals to detect potentially deadly seizures and alert wearers and caregivers, so they can intervene.

It's official! R Tools for Visual Studio, until now only available as a private preview, is now in public preview and available to everyone as a free, open-source download. RTVS is an add-in for Microsoft Visual Studio, which adds R language development capabilities to the popular Windows-based IDE. If you don't already have Visual Studio, you can download Visual Studio Community for free. Then, download and install RTVS to add the "R Tools" menu to Visual Studio. RTVS in action.

Analytics aren't just for measuring performance; they can also help shape efficient educational institution infrastructures. This involves using predictive analytics to ensure maximum efficiency in the classroom, and if executed properly, the rewards are huge.

Raw data is like unrefined ore: To be valuable, it must be retrieved, cleaned, refined and then delivered to those who find it most desirable.

There are emerging digital tools and platforms that hold big promise for retailers, and these tools all have one thing in common: their use of big data to woo consumers and craft highly targeted merchandise offers.

Vodafone UK's new SIEM system relies on Apache Flume and Apache Kafka to ingest nearly 1 million events per second. In this post, learn about the architecture and performance-tuning techniques and that got it there. SIEM platforms provide a useful tool for identifying indicators of compromise across disparate infrastructure. The catch is, they're only as accurate as the fidelity of the data involved, which is why Apache Hadoop is becoming such a valuable platform for that use case.

List: Key insights from new reports in the big data and analytics market.

It seems like only yesterday when I was listening to university professors say at a media luncheon at Strata that they weren't really sure what skills a data scientist needed, since the role was so new, nor what to teach them. That was three or four years ago or so. Since then, the role has been much better defined, though still evolving. Enough time has lapsed that now there's even a scorecard for big data graduate programs. And here are the top 50.

Continuum Analytics is set on turning scientists, physicists, mathematicians and engineers into data scientists as fast as possible. That's why they launched their Anaconda Skills Accelerator Program, the acronym for which, ASAP, drives home the need for speed in filling the data science skill gap.

MapR announced added capabilities to its Converged Data Platform for Docker, including persistent storage and integrated resource management. Essentially, the platform acts as a comprehensive data services layer for Docker containers.

The White House launched a new open data effort called The Opportunity Project designed to open and expand opportunities at the community level through the use of readily available, user-friendly federal and local data. Through data visualization, civic leaders, community organizations, families and individuals can readily identify "critical resources such as access to jobs, housing, transportation, schools, and other neighborhood amenities." Civic leaders can also readily see where inequality exists in access to resources and take corrective measures.

Microsoft will be celebrating Thursday the launch of SQL Server 2016 at its Data Driven event in New York. Among many new advancements and improvements to SQL Server this go around is the extension of its capabilities to run on Linux.

KPMG, the global global tax auditing and advisory firm, aims to use Watson's cognitive computing capabilities to look at volumes of data that, from a human standpoint, would be impossible to manage.

CEO says in code fights, you get matched up against one person or a group of people and are timed to try to accomplish a task better than the others.

Facebook's open networking technology is making inroads into the data center establishment, with global giant Equinix adopting the company's Wedge switch design and an open-source architecture in some of its facilities. The collaboration is the latest sign that network and server designs coming out of the Open Compute Project, which Facebook launched in 2011, are entering the IT mainstream. It was announced at the OCP Summit in San Jose, California. OCP promotes open-source hardware that any manufacturer can make, bringing some of the efficiencies of Web-scale infrastructure built in-house at places like Facebook to general enterprises. Lower costs and greater flexibility are the key advantages that fans ascribe to this approach.

Machine learning is an obvious complement to a cloud service that also handles big data. Often the major reason to collect massive amounts of observables is to predict other values of interest to the business. For example, one of the reasons to collect massive numbers of anonymized credit card transactions is to predict whether a new transaction is valid or fraudulent with some likelihood.

Microsoft has released the next version of its Dynamics AX enterprise resource planning software, giving companies a path to running more of their businesses in the cloud. The update, made available in Microsoft's cloud on Tuesday, has a new design intended to make the software easier to use, so people can get more work done. It's all run through a browser-based portal, so people can access it wherever they are, and on any sort of device, whether that's a desktop PC, a smartphone or something in between. The on-premises version of the suite will be released at a later date. Dynamics AX also connects with Microsoft's Power BI to do data visualization, which means users can get an easy, at-a-glance look at key business metrics, and implement custom visualizations for understanding data. 

Analysis: Could the EU's 2018 smart car technology requirements prove too much for Brits?

Google DeepMind's AlphaGo artificial-intelligence system has won the first game in a cliffhanger Go match with top player Lee Se-dol, raising expectations afresh about whether machines can be programmed to overtake humans in intellectual capabilities. The match has been billed as a major challenge by a computer in a game of tremendous complexity, in the footsteps of the now legendary chess victory of IBM's Deep Blue against Garry Kasporov in 1997 and the 2011 win in the Jeopardy quiz show by Watson, another computer from Big Blue.  AlphaGo, playing through a human assistant to whom it prompted moves from a computer screen, won  Wednesday the first of five games it is playing in Seoul, South Korea with Lee, after the player resigned.  The winner of the match stands to gain US$1 million in prize money, which Google DeepMind has promised to donate to charity if AlphaGo wins.

Google today launched a new search feature designed to help you plan your next vacation. But there's a catch: Destinations on Google only works on mobile devices. This is by design. Google found that in 2015, the company's search engine saw a 50 percent increase in travel-related questions on mobile phones. And so it set out to solve the problem of how to answer the various vacation questions (Where? When? How much?) on a small screen. Here is how it works. Google a continent, country, or state on your phone and append the word "destination." You'll get back a slew of information. The Explore tab shows you year-round weather expectations and when your destination is most popular, based on historic visits from other travelers.  

Tonight YouTube will be live-streaming the first of a five-game match of the ancient Chinese board game Go between the AlphaGo artificial intelligence system from Google's DeepMind AI research lab and Go superstar Lee Sedol. YouTube will be live-streaming the four subsequent matches over the next few days. The match is reminiscent of IBM's Watson supercomputer taking on (and beating) contestants Ken Jennings and Brad Rutter on the game show Jeopardy, except that Go is very challenging, much more so than chess, and AlphaGo's underlying technology is considerably more complex than Watson's.

The Internet of Things is at the heart of the digital transformation we are witnessing in society today. The advancements of this connectable ecosystem bring several important considerations including data security and privacy and the elements of the infrastructure. In this second installment of a multipart series, see why the core focus for enterprises revolves around perspectives on key components of the architecture, and take a look at some cool implementations in healthcare and urban living.

Financial institutions of all kinds are seeking ways of catering to the influx of millennials who are filling workplaces at record numbers. Tune in as Jon Stein, CEO of Betterment; Priya Malani, cofounder of Stash Wealth; Carolyn Baird, noted author of papers analyzing the millennial phenomenon; and Brian Walter of Watson financial services join Alex Baghdjian and Rob Stanich in discussing how millennials are managing and investing their money.

Identity management doesn't have the cachet of robots that automate mundane tasks, sensors that shuttle data between machines or analytics that ferret out insights. But as businesses increasingly rely on digital technologies for growth, ID management technologies are crucial for tracking consumer interactions with your brands online. It is, in fact, the digital handshake that grants customers passage into your website, making it possible to tailor content, make recommendations and offer other perks to retain customers. American Cancer Society CIO Jay Ferro

The Opportunity Project signals a shift in the Obama administration's approach to open data. Read about the thinking behind this new initiative.

If customers have a poor experience with a brand, they don't care who or what may be the cause; it is the brand that will take the reputation hit.

Biometrics has to confront some significant challenges before it becomes the security technology of the future.

Big data analytics is one of the fastest growing fields in enterprise technology. Here are the best places to study in the field.




Food poisoning can strike anywhere hygiene standards are lax, but researchers have developed a new app that uses machine learning to help minimize the number of people affected. One out of every six U.S. residents gets food poisoning each year, and when they do, many of them write about it on Twitter. That's where nEmesis comes in. Developed by computer-science researchers from the University of Rochester, the software uses natural language processing and artificial intelligence to identify food poisoning-related tweets, connect them to restaurants using geotagging and identify likely hot spots.

What an exciting time for Hadoop, for the Community and for Hortonworks. Last week, we announced our strategy around Open and Connected Data Platforms. And followed-up with the latest release of our flagship product, the Hortonworks Data Platform 2.4. This included the release of Apache Ambari 2.2, which will further enable enterprises to harness the power of data by taking the guesswork out of provisioning, managing, monitoring and securing Hadoop.

In recent years, customer loyalty has become even more important to airlines' success. By using data analytics to understand customers' needs, airlines can determine how to stand out from competitors and offer meaningful customer experiences based on individual preferences.

When floods, fires and other emergencies strike, natural disaster management teams respond with technology on their side. Data analytics is changing the traditional approach to rescue and recovery, one set of information at a time.

If you're searching for tools that can help you prototype commercial Internet of Things (IoT) solutions for the modern market, have a look at the Qualcomm DragonBoard 410c. This tiny package offers a wide array of capabilities designed to help you uncover the insights that await you within your data, aided by the power of the IBM Watson IoT platform.

The enterprise is anxious to push more of its workload onto the cloud but still has trust issues with the adoption of third-party infrastructure.

Today our guest blogger is Keith Manthey, CTO from EMC. As part of my job, I regularly meet with clients around their Apache Hadoop journey. I often meet executives after they have encountered a catalytic event. In one particular meeting I vividly remember, the client had suffered over 24 hours of downtime on their Hadoop cluster. The second question out of their mouths was how were they able to provide four 9s availability (or 99.99%) uptime to their Hadoop environment.   This conversation track usually leads to a very interesting discussion around the operational sustainability of their analytics platforms in which Hadoop is their bedrock. Last week, I was very excited to read the announcement of general availability for Hortonworks Data Platform 2.4 that includes community innovations of Apache Ambari 2.2, as well as Apache Spark 1.6.  

This entry was posted in News and tagged , , , , , , . Bookmark the permalink.