Big Data News – 12 Oct 2015

Top Stories
The Obama administration will not pursue legislation that would compel companies to decrypt customer data, but will continue to try to persuade them to give them access when needed.

Dell's plan to acquire EMC for $67 billion reflects the major disruptions taking place in enterprise IT as businesses increasingly shift toward cloud computing. Here's what IT leaders need to know about the deal.

Researchers at Harvard University's medical school and other institutions studying post-traumatic stress disorder and other mental health issues plaguing American war veterans report they have developed an "actuarial model" based on machine learning that can help predict future violent crimes by U.S. soldiers. The researchers noted that their model based on an administrative dataset of more than 975,000 U.S. soldiers could help pinpoint those most prone to violent crime. That information could then be used to make existing interventions more effective.

Solix Technologies, Inc., a leading provider of Enterprise Data Management (EDM) solutions, announced the Standard Edition of the Solix Big Data Suite. Solix Big Data Suite Standard Edition is a free Information Lifecycle Management (ILM) framework, built on a single node cluster and packaged in a virtual environment.

CMOs have a lot to think about in terms of adopting evolving marketing technology. As technology becomes ubiquitous across organizations, CMOs need to not only be aware of the latest data-driven technologies, but also have the right employees to ensure they're using the technology to benefit the company. Not surprisingly, according Chris Chodnicki, CTO and co-founder of R2integrated, a marketing agency focused on helping companies achieve campaign goals through cloud and data strategies, one of the biggest things CMO's will need to focus on is data.

Don't hold on to manual process that throw doubt on your data and put you at risk of regulatory noncompliance. Instead, discover how adopting an insurance software solution can help your company boost its operational efficiencies, enhancing its ability to compete in the marketplace.

As the ancestral home of Hadoop, Yahoo is a big user of the open source software. In fact, its 32,000-node cluster is the still the largest in the world. Now the Web giant is souping up its massive investment in Hadoop to give it a deep learning environment that's the envy of the valley. With more than 600 petabytes of data spread across 40,000 Hadoop nodes, Yahoo is obviously a big believer in all things Hadoop.

The combination of Hadoop, Spark and Kafka is creating the foundation for building more agile data warehouses.

A new month brings a new contest winner. Meet Andreas, the winner of Contest 9 (also known as the October Prize)! Andreas originally hails from Sweden, then moved to the United Kingdom for his university studies. In the UK, he studied mathematics and then stayed to pursue a career in the finance industry, before embarking on a graduate degree in mathematical physics. He is currently a PhD student in Spain continuing his journey in mathematics. Andreas stumbled across Quantopian while traversing the web, and was immediately hooked. With no previous background in Python, he started learning how to create trading algorithms.

MuleSoft announced that its new Anypoint B2B integration framework can be used to invoke EDI transactions using REST APIs.

As more industries take advantage of the Internet of Things (IoT), higher education risks being left behind. However, integrating educational institutions and processes into the IoT offers new learning opportunities for students and more efficient management methods for school administrators.

by Dan Woods An analytic infrastructure can be much like Mark Twain's definition of a classic: "A book which people praise and don't read." An analytics platform is often referred to but rarely architected. Business analysts and data scientists often talk about the power of analytics without talking about the end game. But to make any progress in the big data world — and remain competitive — companies must change the way they think about analytics and implement an analytic infrastructure. The current approach to big data analytics is simply unsustainable.

Recent innovations in the Internet-enabled Connected Cars that we drive today have spawned a whole new set of opportunities and challenges for carmakers. The opportunities come from the ability to capture detailed, current data on how drivers actually operate their cars and how those cars respond to that use. Register for the October 22 Webinar That data can be extraordinarily valuable for uses such as preventative maintenance, product development, manufacturing optimization and recall avoidance.

While you're at IBM Insight 2015, hit the bookstore for more on a variety of topics covered in regular Insight sessions. If you like, meet the authors to get your book signed and take a moment to chat. Don't miss this chance to bring your learning home from Insight.

The battle of privacy versus data and analytics continues to rage in the public sector. The axiom "all human beings have three lives: public, private and secret" may need to be modified, as today's data and analytics likely mean just one life for private citizens –a public one.

Powerpoint is a powerful application for creating presentations, and allows you to include all sorts of text, pictures, animations and interactivity to create a compelling story. Most of the time you'll use the Powerpoint application to create slides, but if you want to include data and/or charts in your slides, in the interests of reproducibility you may want to automate the slide creation process.

Powerpoint is a powerful application for creating presentations, and allows you to include all sorts of text, pictures, animations and interactivity to create a compelling story. Most of the time…

With backing from former US CTO Aneesh Chopra, Apigee is trying to make healthcare access easier. The API developer is rolling out a new platform that helps bridge some of the interoperability gaps when it comes to electronic healthcare records.

Hollywood is notorious for remaking films that made big money in the past rather than taking a risk on producing too many new movie scripts. Sure, the remakes are juiced up and refreshed in a myriad of ways to make an old story more palatable to modern audiences, but face it, they're still serving leftovers. Unfortunately, too many companies in other industries are doing the same thing with big data.

When considering data integration and governance, handling data is akin to dealing with a cluttered desk. Take a look at some tips that can apply to both tasks. Maybe now is a good time for some fall organization?

Most print media companies have struggled to make money in the 21st century, but The New York Times is using predictive analytics tools to gain a competitive edge.

Figshare, an online digital repository for academic researchers, announced its next-generation data management platform for researchers. The targeted market includes individual researchers, teams in any-sized organization, funders and publishers. Its capabilities offer better control and data discoverability plus enhanced data collection, data sharing and researcher collaboration and alerts.

Mark Reinhold, Chief Architect of the Java Platform Group at Oracle, published a report on the State of the Module System with an emphasis on what the objectives are (and aren't) and an explanation of how these are currently met. The publication has triggered comments among users on the apparent overlap with existing frameworks like OSGi. InfoQ looks at background and current state.

Alteryx teamed with Microsoft to create analytic workflows that output datasets directly to Microsoft Power BI for business analysts, the company announced Thursday. The partnership also enables deeper integration between Alteryx Analytics and Microsoft SQL Server (2008, 2012, 2014).

Financial executives face many challenges, but accurate and timely financial reporting is one of the most daunting tasks. Analytics can improve the quality of financial reporting and provide insights for financial decision making.

Many have recognized that the recent decision by the European Court of Justice (ECJ) on Safe Harbor and the flight…

According to a new survey from CompuCom, 48 percent of companies are using big data analytics to improve their overall customer experience, 32 percent are using them for information security and 20 percent are using them for data warehouse optimization. But is it working?

Latvian-based NoSQL cloud database vendor Clusterpoint launched Clusterpoint 4 Thursday, a computing engine that combines an instantly scalable document-oriented database with a computing model. In other words, the company is seeking to disrupt the cloud database market by unifying computational power and data as well as changing the pricing model.

Data reservoirs are useful tools that can help organizations create new products and services, increase customer service and efficiency, and reduce waste and fraud. But what is a data reservoir and how do you create it?

Measuring and sustaining customer loyalty is critical in today's ever-changing consumer marketplace. Surveys can provide helpful information and insights, but they only capture sentiment at one specific point in time. Plus, a high satisfaction rating on a questionnaire does not necessarily equate to brand loyalty.

AWS announced a bunch of new services at Re:Invent, but more noticeable was the revelation of a more assertive Amazon.

In this article, Seth DeLand and Adam Filion use MATLAB to complete the entire data analytics workflow for a load forecasting application. Using this application, utility analysts can select any region in the state of New York to see a plot of past energy load and predicted future load.

In this special guest feature, Bruce Reading, President and CEO of VoltDB, discusses the importance of capturing value from your data long before it goes dark.

Accelerite, a provider of infrastructure software for cloud, mobility and endpoints and a Persistent brand, announced today its entry into the Internet of Things via a new platform called Aepona IoT. The product is designed to encompass the whole of IoT service enablement from creation to execution and from supporting developer APIs and monetization to onboarding, connectivity and analytics.

The idea started several years ago when I signed-up for Aeroplan's digital insight community, which then spawned the conversation here at SAP… why don't we do something like this with our customers? The proposition is simple. We want to hear exactly what individual users think about very specific topics. And we want to make it…

The book Software Development Metrics by Dave Nicolette explores how to use metrics to track and guide software development. It explains how different development approaches and process models, like traditional waterfall-based or iterative agile software development, affect the choice and usage of metrics. It describes metrics that can be used for steering work and for managing improvement.

ResponseTek is a software vendor whose platform and services help companies collect and act on feedback from their customers. It supports a closed-loop process that collects feedback, analyzes it, provides customizable reports and analysis dependent on the user, and most importantly enables taking action based on the information.

Modern cloud connectors are designed to connect via loosely coupled interfaces that allow cloud systems to share data in a flexible manner. The research thus suggests that for organizations needing to integrate data from cloud-based data sources, switching to modern integration tools can streamline the process.

Ed McCann, Andrew Taylor and Deb Oxley (moderator) discuss the challenges and obstacles their companies faced in becoming employee owned, as well as the benefits and rewards.

BigPanda, the data-science platform that turns noisy information technology (IT) data into actionable insights for businesses, announced it has raised $16 million in Series B financing led by Battery Ventures, with participation from existing investors Sequoia Capital and Mayfield.

Alexander Stigsen explores the challenges & opportunities in developing for the new mobile cloud, where most of the processing power is not in a datacenter but in the pockets of users.

Sometime in the 2030s, NASA wants to land astronauts on Mars. Here's a look at what it will take to get us there.

Josh Bregman explores some of the unique security challenges created by both the development workflow and application runtime, explains why and how the current approaches in SecDevOps 1.0 are insufficient, and how SecDevOps 2.0 techniques including Software Defined Firewalls (SDF) provide a promising path forward for all parties involved.

Columnar data storage can offer significant performance improvements over the way database tables are traditionally stored, but they aren't always faster. Aleksandr Shavlyuga explores the power, and limitations of SQL Server's ColumnStore Indexes. By Aleksandr Shavlyuga

1. European Union data sovereignty laws have long had a "Safe Harbour" rule stating it was OK to ship data to the US. Per the case Maximilian Schrems v Data Protection Commissioner, this…

Viktor Gamov covers In-Memory technology, distributed data topologies, making in-memory reliable, scalable and durable, when to use NoSQL, and techniques for Big In-Memory Data.

Daniel Seltzer discusses what intellectual skills are needed to be able to build and lead a successful group. This requires a new set of capabilities that aren't taught in school and don't come from certification programs. Daniel focuses on a concrete set of skills that you can and should develop for your own benefit and that of any group of people you come to lead. By Daniel Seltzer

Sherif Mansour shares from his experience at Atlassian building simple products using Agile product requirements, prototypes, customer interviews, and user journeys. By Sherif Mansour

WANdisco, a leading provider of continuous-availability software for global enterprises to meet the challenges of Big Data, announced that a leading 24-by-7 provider of financial data will use WANdisco FusionT to deliver continuous availability and performance for its predictive customer analytics applications built on the Hortonworks Data Platform (HDP).

One year after hitting 1.0, Elixir 1.1 is out. It brings new public APIs, performance improvements, and tooling improvements. InfoQ has spoken with Jose Valim, Elixir's creator.

In this special guest feature, Robert Buck, VP of Technology at Deep Information Sciences makes the case for new adaptive technology that couples databases and machine learning to address the demands of a data driven economy.

MemSQL, a leader in real-time databases for transactions and analytics, announced that Teespring, a leader in on-demand social commerce, selected and deployed MemSQL to optimize its sales analytics and create a seamless experience for apparel buyers and sellers.

Based on the amount of retailers that have been forced to shut their doors in 2015 alone, there is a major shift that is moving through the retail industry at full force. (Here's the list of stores closed so far this year.) We've been talking about this for a while, and now retailers are feeling… The post Using Predictive Analytics to Bring Retailers Closer to Their Customers appeared first on Predictive Analytics Times.

We're proud to announce at the AWS re:Invent conference that we have extended our Big Data solutions to Hadoop in the cloud. Attunity CloudBeam now facilitates data transfer between on-premise and…

Jamie Raines was born a girl, but at age 18 he began a three-year journey to transform his body to match his identity with hormone therapy. Every day Jamie took a selfie to document the change, and…

The consumer products industry is undergoing a seismic shift in which markets are becoming increasingly fragmented. Thanks to technology, the process of buying is becoming synonymous with research; consumers are the masterminds of finding their way to great products and even better value. It's this context that creates a need for CPG companies to invest in new brand marketing and loyalty programs.

Data,data,data everywhere and what do I do with it. How do I make sense of it that is useful to the business? More importantly, how do I tell the story now that the solution is built? VISUALIZATION. Virtually any software package today has some kind of visualization, even rudimentary tools such as Excel. But the… The post Visualization: Panacea for Building Analytics Solutions? appeared first on Predictive Analytics Times.

Alteryx Inc., a leader in data blending and advanced analytics, announced a new relationship with Microsoft Corp., to provide business analysts a better way to take advantage of data blending and advanced analytics that leads to deeper insights in hours, not the weeks typical of traditional approaches.

Amazon has announced QuickSight at AWS Re:invent conference. QuickSight a complete Business Intelligence solution to help customers gain insights from the data they have stored in AWS.

A major transformation has shaken the media and entertainment business over the past 10 years. It's hard to imagine another industry that has experienced the same major shift. Disruption is a natural part of business, of course, but the pace of change in media and entertainment trends has been breathtaking.

Segment, the customer data hub, today announced $27 million in Series B investment, led by Thrive Capital with participation from existing investors Accel Partners, Kleiner Perkins Caufield & Byers, and Jon Winkelried, former president of Goldman Sachs.

Amazon launched its IoT at its Re:Invent conference in Las Vegas Oct. 8, and illustrated how it will handle data pouring into its storage systems.

Exclusive Insights From Top Data Scientists You can't afford to fall behind on data trends. And your calendar is jammed. What's a data scientist to do? The 2015 PARTNERS Virtual Conference enables you to watch and participate inthree days of sessions covering best practices from top companies including Monsanto, Target, Symantec, U.S. Air Force and McKinsey. From IoT to architecture to data lakes, you'll have insight into the latest trends and control over your schedule — live as they happen or on demand up to 90 days later. 

In case you missed them, here are some articles from September of particular interest to R users. A tutorial on using R with Jupyter Notebooks and how to control the size of R graphics therein. A new…

Enterprise Predictive Analytics: Success Stories Join us for the latest DSC Webinar on October 27th, 2015 Space is limited. Reserve your Webinar seat now   Join us for our latest DSC Webinar series to learn how to better access your data, provide more effective business intelligence and systems integration, improve distribution management and demand planning as well as promotional pricing.

Here's a look at the top 10 emerging IT trends from the Gartner Symposium 2015, and what they mean for your IT operation.

Data shows that organizations spanning many industries can transform business and enhance growth by using cloud-based delivery models for applications and streaming analytics. But not every business scenario is well suited for services through the cloud. Take a look at five signs that an organization may benefit from a cloud-based streaming analytics service.

I'll be teaching two hands-on labs at Insight 2015 in Las Vegas: LCD-3459 Introduction to Data Science Data science is a very popular job profile and in great demand in a wide variety of…

By connecting devices into ecosystems, application programming interfaces are broadening the horizons of the Internet of Things, creating previously unimagined opportunities. By using APIs to overcome security and interoperability challenges, we can begin to harness the true potential of the Internet of Things.

Frustrated with a fragmented system that doesn't give caregivers or patients access to full patient data, the CIO of Fairview Health Services has the beginnings of a plan.

Today's job market for highly skilled employees is white hot. The workforce is quick to move from one opportunity to another, and employers are faced with high costs for recruiting and reduced inefficiencies resulting from having to constantly train new employees. The exact cost of turnover is hard to estimate, since it varies significantly among different…

Weather is both asset and liability to energy providers. Even in a world of renewable energy, adverse weather events can create power outages on a citywide scale. Using weather data analysis, energy providers can forecast outages with high levels of reliability, then prevent or mitigate them ahead of time.

Chris Richardson explains the appeal of Scala, functional programming in Java and other languages, the basics of Event Sourcing, and his perspective on the state of the Java ecosystem.

Amazon's QuickSight provides fast and easy-to-use analytics and visualizations that can draw on multiple data sources including AWS-based services and external platforms such as Salesforce.

KNIME offers open source data analytics, reporting and integration tools, as well as commercial software that can help build more efficient workflows.

Treasure Data, a leader in analytics for large-scale event data, unveiled a new set of enterprise-level integrations with Amazon Web Services, Salesforce and Marketo, along with new features to simplify event data analytics management.

A massive surge in unstructured data creates tough challenges for IT departments. The average enterprise will need to manage 50 times more data by 2020.  Originally posted on

As organizations struggle to manage growing silos of information, data blending provides analysts with a more streamlined way to collect and analyze this data.

In the first four installments of this series, we reviewed new and enhanced frameworks included with iOS 9 SD, changes to Swift and Objective-C, and the new Safari content blocking API. In this article, we will describe what is new within Apple Developer Tools, including Xcode Playgrounds, LLDB, UI testing, Interface Builder, etc.

To track the mobile traffic on your website, it is recommended to create Google Analytics mobile dashboard. Google analytics dashboard provides an easy way through which the website owner can analyze and the share the data obtained. Essentially, two types are available for startups; mobile traffic dashboard and mobile vs. desktop vs. tablet dashboards.

Google has open sourced the specification for a restricted HTML that is meant to improve the mobile experience on the web.

The Snowden revelations and the emergence of ‘Big Data’ have rekindled questions about how security practices are deployed in a digital age and with what political effects. While critical scholars have drawn attention to the social, political and legal challenges to these practices, the debates in computer and information science have received less analytical attention. This paper proposes to take seriously the critical knowledge developed in information and computer science and reinterpret their debates to develop a critical intervention into the public controversies concerning data-driven security and digital surveillance.

A cornerstone of the semantic web is its use of newer graph-based approaches and technologies — such as the RDF and SPARQL W3C initiatives. Given the internet is a giant web of connected data this model works well compared to traditional relational techniques where it has been necessary to structure data in ways less geared to showing complex relationships such as hierarchies.

Cox Automotive has a lot of Agile teams across its 20+ brands and companies. In recent years, it became clear that they needed to bring together Agilists from across the enterprise to connect, share and learn. So they decided to organize their own, company-internal Agile Open conferences. Now approaching their 3rd year, these events have been quite successful and really brought people together.

Your new IT strategy has been accepted by the board. It WILL happen. But how quickly it beds down and shows real benefits depends on everyone in the organisation – regardless of their role – being ready to accept it.

Find out the two things that make the Data Distribution Service standard unique, and why one CEO says the technology's potential is "virtually limitless."

Keith Dahlby overviews OWIN, discussing its implications for .NET web application design and reviewing a real-world example of OWIN in action.

Erik Dahl shares a number of lessons learned along the way, outlining a set of UX Axioms designers and developers alike can use to integrate UX into their practice.

IoT is currently a Babel of disparate "standards," but a new initiative from Samsung promises to change that.

This entry was posted in News. Bookmark the permalink.