Big Data News – 09 Aug 2016

Today's Infographic Link: Top Five Medical Discoveries

Featured Article
I've often heard of organisations that say they're going to "try Hadoop". That's great that you're going to see what it can do for your business and what it is capable of, but Hadoop is one of those technologies where if you don't commit time and resources to it, then it's going to be a failure. And I can virtually guarantee that success won't come easily. But don't let that deter you from jumping in. But read on for some helpful hints.

Top Stories
With only three weeks to go until Hadoop Summit Melbourne, anticipation is building for what looks set to be a fantastic event for the Hadoop community.   Whether it's pre-event training, our keynotes, sponsor sessions or the 3-4 business or technical sessions you have to choose from every hour, your agenda is undoubtedly going to be…

Internet of Things Expo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 19th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago.

Data centers can and often are costly to run. IT and business executives are starting to realize that hundreds of thousands of dollars can be saved in energy and operational costs through improved physical infrastructure planning, minor system reconfiguration and small process changes.

Generally speaking, there are two kinds of companies in the world: data rich and data poor. The richest of the data rich are easy to name: Google, Facebook, Amazon, Apple. But you don't need to be at the top of this list to use data to create value. You need to have the tools in place to turn information (data) into action. That's what the data rich do that the data poor and the data middle class do not.

The Data Lab has partnered with New York's globally renowned, The Data Incubator (whose courses are reputedly harder to get into than Harvard), to develop the three-week data Boot Camp as part of a drive to plug the nation's data skills gap. It is aimed at helping to unlock the economic potential of data to Scotland, estimated to be worth ?17 billion* in Scotland alone.

One of the things that I love to be able to do in my role as a Product Manager is spend time with customers and prospects understanding what is important to them as they make their journey to a modern data architecture. Based on these discussions I've noticed a consistent set of themes emerging; themes that span industries, use cases, and geographies. I've come to think of these themes as principles of an Enterprise Data Architecture.

News: Open Banking programme will come into force by 2018.

We Do Sky is also developing an extensive list of industry-specific tools to help drone pilots generate intelligence for multiple industries, Makkar said.

Genome-wide association studies, which try to find correlations between particular genetic variations and disease diagnoses, are a staple of modern medical research. But because they depend on databases that contain people's medical histories, they carry privacy risks. An attacker armed with genetic information about someone — from, say, a skin sample — could query a database for that person's medical data. Even without the skin sample, an attacker who was permitted to make repeated queries, each informed by the results of the last, could, in principle, extract private data from the database.

The fashion portal claims to have achieved a 95% accuracy rate through this simpler level of image-mapping technology.

An artificial intelligence initiative launched last week seeks to weave predictive AI technology into emerging health and "wellness" products. According to initiative organizers, medAI seeks to move beyond "button pushing and impersonal automation scenarios" to bring a new level of AI-powered personalization to scenarios such as reminders to take medications or check blood pressure.

The big move is Amazon's beating Exxon Mobile (used to be number 1 for many years) to the fourth spot. The switch came after Amazon posted its fifth straight quarter of profits last week as the oil giant's profits tumbled 59 percent during the same rough period. If Exxon continues its drop, then Facebook will beat it in days.

Kubernetes, Docker and containers are changing the world, and how companies are deploying their software and running their infrastructure. With the shift in how applications are built and deployed, new challenges must be solved. In his session at Dev Ops Summit at19th Cloud Expo, Sebastian Scheele, co-founder of Loodse, will discuss the implications of containerized applications/infrastructures and their impact on the enterprise. In a real world example based on Kubernetes, he will show how to migrate an existing application to Docker and Kubernetes, and what the benefits are.

Pokemon Go has stormed onto the scene this past week and the buzz is everywhere about it! However, being so successful in such a brief period of time can have a downside. Reports of players being unable to access services started appearing in relatively short order. Amazon's CTO jumped into the fray offering their cloud services to help scale NiantecLab's (creators of Pokemon Go) server-side infrastructure. While the Pokemon Go app has been positively rated by the majority of players, there are clearly a large number of players who thought differently, as indicated in the Google play App ratings.

Hadley Wickham's dplyr package is an amazing tool for restructuring, filtering, and aggregating data sets using its elegant grammar of data manipulation. By default, it works on in-memory data frames, which means you're limited to the amount of data you can fit into R's memory. Hadley also provided an extension mechanism to make dplyr work with external data sources, and so Hong Ooi created the dplyrXdf package to work with Xdf data files.

If Hadoop is the tool that ushered in the era of big data, Spark is the one that's driving the next phase of its evolution. Spark is the brainchild of a group of Berkeley grad students, and brought an entirely new set of use cases for big data and data analytics. Spark is more than just the right thing at the right time, it's a way to get speed out of analytics where before there was power, but nothing in the way of real-time.

DRaaS, whether or not it is the same or slightly different than DR/BC, is growing.

The numbers tell the story. High costs and poor rankings continue to plague telecommunications organizations and their traditional customer care call center operations. See how two key areas of the brain–one that controls reactive response and one that orchestrates proactive response–may be the key to transforming customer care in the telecommunications industry.




Ugh. I see so many organizations get so close to the goal line with Big Data, and then get sidelined by something that is not nearly as important to the business. It is easy to see how these organizations get distracted as they get near the Big Data goal line, because the average CIO and Line of Business executive are continually fighting battles. They are so busy fighting battles that they forget to focus on winning the war.

Bluetooth 5, married with the IoT, will revitalize this strangest of protocols, which manages to hide in plain sight.

DNS is a core component of the Internet. This rather complex system exists to perform the basic function of translating website Domain names (ex: Catchpoint.com) into an IP address. It acts as the liaison between the language that users can understand, and the address of the data center or destination in which the site is being hosted. DNS is what shapes the modern Internet. It enables users of all skill sets and technical knowledge levels to use this incredibly advanced resource on a daily basis. Businesses that exist entirely on the Internet like online banking and ecommerce companies have not only emerged into the market, but have steadily gained a global presence.

To keep up with data growth, companies need to combine on-premises storage performance and availability with the elasticity and economics of the cloud.

During the last two years, Microsoft has upped the ante on Machine Learning and Analytics. From hiring top notch data scientists to acquiring niche startups, Redmond has made the all the right moves to transform Azure into one of the best analytics platforms. These investments have started to pay off for the company. It has been successful in articulating and demonstrating the value of data-driven insights to governments, medical institutions, and public sector organizations.

Google has acquired Orbitera, a startup that aims to make it easier for software vendors to sell cloud-based products to businesses. The startup gives software vendors a suite of tools for deploying and managing cloud applications and for billing businesses that use them. Orbitera supports deploying applications on Amazon Web Services and Microsoft Azure, not Google Cloud Platform.

Infrastructure complexity is not unique to enterprise datacenters. Just because a business or organization is small does not mean it is exempt from the feature needs of big enterprise datacenters. Small and mid-size organizations require fault tolerance, high availability, mobility, and flexibility as much as anyone. Unfortunately, the complexity of traditional datacenter and virtualization architecture hit the SMB the hardest.

While a computer that could predict your future is tantalizing. Would you want a glimpse into your future, whether it be good or bad? Until recently, most medical statistics have been bell curves created in hindsight. We know from studying the U.S. population that nearly 1 in 10 people have diabetes. We even know some…

Google has acquired Orbitera, a startup that aims to make it easier for software vendors to sell cloud-based products to businesses. The startup gives software vendors a suite of tools for deploying and managing cloud applications, and for billing businesses that use them. Right now, Orbitera supports deploying applications on Amazon Web Services and Microsoft Azure, not Google Cloud Platform.

Is there a next phase for cloud computing? During the past few years, cloud computing has become a mainstream element of modern software solutions just as common as websites or databases.

Qlik® (NASDAQ: QLIK), a leader in visual analytics, today announced the expansion of the Qlik Academic Program with its new Data Analytics Curriculum.

As your data-driven organization considers incorporating new data sources like mobile apps, websites that serve a global audience, or sensor information from the internet of things, technologists will have questions about the required attributes of a streaming data platform.

Without careful coordination, cloud environments can introduce all manner of bottlenecks that inhibit the smooth flow of data.

Customers today expect more from their banks–personalized service, real-time solutions and the ability to do business when and how it is most convenient for them. To meet these demands, you need the ability to deliver added value at every opportunity through highly personalized and timely offers and interactions.

StackIQ is a California based company that offers a server automation platform for clustered, scale-out IT infrastructure. I met up with Greg Bruno, VP of engineering and co-founder of StackIQ, to learn more about the company, its product and the Stacki open source project. Here is an edited version of that interview. What is StackIQ?

Canvas 8 lets end users automate a paper-based process without intervention on the part of the internal IT organization or a third-party developer.

In 2016, the need for security professionals has taken on a new sense of urgency.

The speed at which data is generated, consumed, processed, and analyzed is increasing at an unbelievably rapid pace. Social media, the Internet of Things, ad tech, and gaming verticals are struggling to deal with the disproportionate size of data sets. These industries demand data processing and analysis in near real-time. Traditional big data-styled frameworks such as Apache Hadoop is not well-suited for these use cases.

It has been another exciting week on Hortonworks Community Connection HCC. We continue to see great activity and recommend the following assets from last week. Top Articles from HCC Implementing a real-time Hive Streaming example by:mjohnson The Hive Streaming API enables the near real-time data ingestion into Hive.

Last week, we reported our Q2 2016 results as well as the news of Herb Cunitz transitioning out of Hortonworks to pursue his next career path of being a CEO. I want to personally thank Herb for his contributions to Hortonworks. Herb has been a terrific partner of mine for over a decade and across several… The post The Journey Ahead appeared first on Hortonworks.

Analysis: What are the popular trends driving uptake of languages?

Apache NiFi to prioritize which images should be sent to Spark in the cloud for computer vision machine learning During the 2nd demo of the Data Hacks & Demos session, at Hadoop Summit San Jose, Simon Ball demonstrated how to take data received from the edge, and run facial recognition on a more powerful cloud…

In anticipation of his upcoming conference co-presentation, Predictive Analytics for Stress Testing – Industry Challenges at Predictive Analytics World Financial in New York City, October 23-27, 2016, we asked Sanjay Gupta, Executive Vice President and Head of Model Development at PNC Bank, a few questions about his work in predictive analytics. Q: In your work with predictive analytics, what behavior or outcome do your models predict? A: We predict defaults of our clients and losses in our portfolio.

Turi, the Seattle-based startup company, is Apple's latest purchase in a string of acquisitions aimed at bolstering its machine learning and artificial intelligence capabilities. These two technologies are changing how CIOs and administrators view new IT investments in the enterprise.

Tools, tools, and more tools continue to explode in the analytics marketplace as being key enablers in facilitating the many tasks and functions of the data scientist. These tools are not only used in conducting advanced mathematical routines and visualization reports but also in creating the analytical file. The use and consideration of certain tools… The post Exploring the Tool kits of Predictive Analytics Practitioners — Part 1 appeared first on Predictive Analytics Times.

The NVM Express interface provides access to shared storage at speeds that are comparable to a locally attached SSD.

The big data ecosystem benefits from a rich and diverse array of products and projects, veritable pieces in a puzzle that IT professionals largely assemble themselves. But this big data abundance can be overwhelming at times, so we put together this guide to help you understand some of the most common terms you'll come across in your big data journey. In the big data jungle, it's worth starting at the top, which brings us to…

Our roundup of intriguing new products from companies such as VMware and Spirent.

Some failing big data projects get millions more pumped into them with the hope that they'll break even. At some point good money needs to stop chasing bad projects, but it takes leadership to make the right decision.

We in IT live in a complex world. While complexity is rapidly increasing, we are working hard on simplifying it. The two things occurring at the same time certainly do not make for a smooth or even linear transition, but at a minimum we are keeping complexity from overwhelming the number of man-hours available to manage the overall IT world.

What does dedication to the client experience look like? To create IBM Bluemix Lift, it meant driving from concept to app in 90 days flat.

In the scope of ecommerce, big data analysis is the collection of large amounts of information in order to determine shopping patterns and insights about customer behavior. Using big data has a lot of added benefits for both the seller and consumer.

This entry was posted in News and tagged , , , , , , , , , , , , . Bookmark the permalink.