Archive | September, 2017

Unlearn to Unleash Your Data Lake

16 Sep

The Data Science Process is about exploring, experimenting, and testing new data sources and analytic tools quickly.

The Challenge of Unlearning
For the first two decades of my career, I worked to perfect the art of data warehousing. I was fortunate to be at Metaphor Computers in the 1980’s where we refined the art of dimensional modeling and star schemas. I had many years working to perfect my star schema and dimensional modeling skills with data warehouse luminaries like Ralph Kimball, Margy Ross, Warren Thornthwaite, and Bob Becker. It became engrained in every customer conversation; I’d built a star schema and the conformed dimensions in my head as the client explained their data analysis requirements.

Then Yahoo happened to me and soon everything that I held as absolute truth was turned upside down. I was thrown into a brave new world of analytics based upon petabytes of semi-structured and unstructured data, hundreds of millions of customers with 70 to 80 dimensions and hundreds of metrics, and the need to make campaign decisions in fractions of a second. There was no way that my batch “slice and dice” business intelligence and highly structured data warehouse approach was going to work in this brave new world of real-time, predictive and prescriptive analytics.

I struggled to unlearn engrained data warehousing concepts in order to embrace this new real-time, predictive and prescriptive world. And this is one of the biggest challenge facing IT leaders today – how to unlearn what they’ve held as gospel and embrace what is new and different. And nowhere do I see that challenge more evident then when I’m discussing Data Science and the Data Lake.

Embracing The “Art of Failure” and The Data Science Process
Nowadays, Chief Information Officers (CIOs) are being asked to lead the digital transformation from a batch world that uses data and analytics to monitor the business to a real-time world that exploits internal and external, structured and unstructured data, to predict what is likely to happen and prescribe recommendations. To power this transition, CIO’s must embrace a new approach for deriving customer, product, and operational insights – the Data Science Process (see Figure 2).

Figure 2:  Data Science Engagement Process

The Data Science Process is about exploring, experimenting, and testing new data sources and analytic tools quickly, failing fast but learning faster. The Data Science process requires business leaders to get comfortable with “good enough” and failing enough times before one becomes comfortable with the analytic results. Predictions are not a perfect world with 100% accuracy. As Yogi Berra famously stated:

“It’s tough to make predictions, especially about the future.”

This highly iterative, fail-fast-but-learn-faster process is the heart of digital transformation – to uncover new customer, product, and operational insights that can optimize key business and operational processes, mitigate regulatory and compliance risks, uncover new revenue streams and create a more compelling, more prescriptive customer engagement. And the platform that is enabling digital transformation is the Data Lake.

The Power of the Data Lake
The data lake exploits the economics of big data; coupling commodity, low-cost servers and storage with open source tools and technologies, is 50x to 100x cheaper to store, manage and analyze data then using traditional, proprietary data warehousing technologies. However, it’s not just cost that makes the data lake a more compelling platform than the data warehouse. The data lake also provides a new way to power the business, based upon new data and analytics capabilities, agility, speed, and flexibility (see Table 1).

Data Warehouse Data Lake
Data structured in heavily-engineered structured dimensional schemas Data structured as-is (structured, semi-structured, and unstructured formats)
Heavily-engineered, pre-processed data ingestion Rapid as-is data ingestion
Generates retrospective reports from historical, operational data sources Generates predictions and prescriptions from a wide variety of internal and external data sources
100% accurate results of past events and performance “Good enough” predictions of future events and performance
Schema-on-load to support the historical reporting on what the business did Schema-on-query to support the rapid data exploration and hypothesis testing
Extremely difficult to ingest and explore new data sources (measured in weeks or months) Easy and fast to ingest and explore new data sources (measured in hours or days)
Monolithic design and implementation (water fall) Natively parallel scale out design and implementation (scrum)
Expensive and proprietary Cheap and open source
Widespread data proliferation (data warehouses and data marts) Single managed source of organizational data
Rigid; hard to change Agile; relatively ease to change

Table 1:  Data Warehouse versus Data Lake

The data lake supports the unique requirements of the data science team to:

  • Rapidly explore and vet new structured and unstructured data sources
  • Experiment with new analytics algorithms and techniques
  • Quantify cause and effect
  • Measure goodness of fit

The data science team needs to be able perform this cycle in hours or days, not weeks or months. The data warehouse cannot support these data science requirements. The data warehouse cannot rapidly exploration the internal and external structured and unstructured data sources. The data warehouse cannot leverage the growing field of deep learning/machine learning/artificial intelligence tools to quantify cause-and-effect. Thinking that the data lake is “cold storage for our data warehouse” – as one data warehouse expert told me – misses the bigger opportunity. That’s yesterday’s “triangle offense” thinking. The world has changed, and just like how the game of basketball is being changed by the “economics of the 3-point shot,” business models are being changed by the “economics of big data.”

But a data lake is more than just a technology stack. To truly exploit the economic potential of the organization’s data, the data lake must come with data management services covering data accuracy, quality, security, completeness and governance. See “Data Lake Plumbers: Operationalizing the Data Lake” for more details (see Figure 3).

Figure 3:  Components of a Data Lake

If the data lake is only going to be used another data repository, then go ahead and toss your data into your unmanageable gaggle of data warehouses and data marts.

BUT if you are looking to exploit the unique characteristics of data and analytics –assets that never deplete, never wear out and can be used across an infinite number of use cases at zero marginal cost – then the data lake is your “collaborative value creation” platform. The data lake becomes that platform that supports the capture, refinement, protection and re-use of your data and analytic assets across the organization.

But one must be ready to unlearn what they held as the gospel truth with respect to data and analytics; to be ready to throw away what they have mastered to embrace new concepts, technologies, and approaches. It’s challenging, but the economics of big data are too compelling to ignore. In the end, the transition will be enlightening and rewarding. I know, because I have made that journey.

Source: http://cloudcomputing.sys-con.com/node/4157284

Advertisements

3GPP Burns Midnight Oil for 5G

10 Sep

Long hours, streamlined features to finish draft. The race is on to deliver some form of 5G as soon as possible.

An Intel executive painted a picture of engineers pushing the pedal to the metal to complete an early version of the 5G New Radio (NR) standard by the end of the year. She promised that Intel will have a test system based on its x86 processors and FPGAs as soon as the spec is finished.

The 3GPP group defining the 5G NR has set a priority of finishing a spec for a non-standalone version by the end of the year. It will extend existing LTE core networks with a 5G NR front end for services such as fixed-wireless access.

After that work is finished, the radio-access group will turn its attention to drafting a standalone 5G NR spec by September 2018.

“Right now, NR non-standalone is going fine with lots of motivation, come hell or high water, to declare a standard by the end of December,” said Asha Keddy, an Intel vice president and general manager of its next-generation and standards group. “The teams don’t even break until 10 p.m. on many days, and even then, sometimes they have sessions after dinner.”

To lighten the load, a plenary meeting of the 3GPP radio-access group next week is expected to streamline the proposed feature set for non-standalone NR. While a baseline of features such as channel coding and subcarrier spacing have been set, some features are behind schedule for being defined, such as MIMO beam management, said Keddy.

It’s hard to say what features will be in or out at this stage, given that decisions will depend on agreement among carriers. “Some of these are hit-or-miss, like when [Congress] passes a bill,” she said.

It’s not an easy job, given the wide variety of use cases still being explored for 5G and the time frames involved. “We are talking about writing a standard that will emerge in 2020, peak in 2030, and still be around in 2040 — it’s kind of a responsibility to the future,” she said.

The difficulty is even greater given carrier pressure. For example, AT&T and Verizon have announced plans to roll out fixed-wireless access services next year based on the non-standalone 5G NR, even though that standard won’t be formally ratified until late next year.

N

An Intel 5G test system in the field. (Images: Intel)

An Intel 5G test system in the field. (Images: Intel)

Companies such as Intel and Qualcomm have been supplying CPU- and FPGA-based systems for use in carrier trials. They have been updating the systems’ software to keep pace with developments in 3GPP and carrier requests.

For its part, Intel has deployed about 200 units of its 5G test systems to date. They will be used on some of the fixed-wireless access trials with AT&T and Verizon in the U.S., as well as for other use cases in 5G trials with Korea Telecom and NTT Docomo in Japan.

Some of the systems are testing specialized use cases in vertical markets with widely varied needs, such as automotive, media, and industrial, with companies including GE and Honeywell. The pace of all of the trials is expected to pick up next year once the systems support the 5G non-standalone spec.

Intel’s first 5G test system was released in February 2016 supporting sub-6-GHz and mm-wave frequencies. It launched a second-generation platform with integrated 4×4 MIMO in August 2016.

The current system supports bands including 600–900 MHz, 3.3–4.2 GHz, 4.4–4.9 GHz, 5.1–5.9 GHz, 28 GHz, and 39 GHz. It provides data rates up to 10 Gbits/second.

Keddy would not comment on Intel’s plans for dedicated silicon for 5G either in smartphones or base stations.

In January, Intel announced that a 5G modem for smartphones made in its 14-nm process will sample in the second half of this year. The announcement came before the decision to split NR into the non-standalone and standalone specs.

Similarly, archrival Qualcomm announced late last year that its X50 5G modem will sample in 2017. It uses eight 100-MHz channels, a 2×2 MIMO antenna array, adaptive beamforming techniques, and 64 QAM to achieve a 90-dB link budget and works with a separate 28-GHz transceiver and power management chips.

Source: http://www.eetimes.com/document.asp?doc_id=1332248&page_number=2

5G use cases

10 Sep
With 5G promising “ultra-high throughput, ultra-low latency transmission, and edge computing”, Huawei and Softbank’s 5G use cases including real-time UHD video, robotic arm control and more.

Seeking their own slices of 5G supremacy, Japan’s Softbank Corp and the Japanese division of China’s Huawei Technologies have “jointly demonstrated various potential use cases for a 5G network.”

As can be seen by the two photos provided at the end of this article, the demonstration “included real-time UHD video transmission using ultra-high throughput, remote control of a robotic arm using ultra-low latency transmission and remote rendering via a GPU server using edge computing.”

In addition, the real-time UHD video transmission demonstrated throughput of “over 800 Mbps.”

The videos show a game of air Hockey being played, with a description of how this works in example 3, below.

The remote control of the robotic arm also demonstrated an “ultra-low latency one-way transmission of less than 2ms.”With SoftBank planning “various experiments to study 5G technologies and endeavouring to launch 5G commercial services around 2020,” it’s clear these kinds of demonstrations are just a glimpse into what is promised to be a glorious 5G future.

Of course, 5G promises to connect everyone to everything, everywhere, especially via a vast array of IoT devices, so security is still a major issue needing to be solved, but as with the final 5G standards, a lot of work is being done in all these regards to deliver solid solutions backed by superior security, and we’re just going to have to wait and see how successful the industry is at these issues.

As for the edge computing mentioned above, Huawei and Softbank state that, “in edge computing, servers are located near by base stations, i.e. at the edge of an mobile network, with a distributed way.”

The dynamic duo state that “This architecture allows us to realise ultra low latency transmission between the servers and mobile terminals. Also, it is possible to process a huge amount of data gathered by IoT devices to decrease the load of the mobile network.”

Here are the demonstration details provided by both companies, with accompanying infographics:

1. Real-time UHD video transmission

“A UHD camera was installed inside the demonstration room to capture outdoor scenery. The data from this camera was then compressed in real-time using an encoder and transmitted through the ultra-high throughput 5G network to a UHD monitor via a decoder, where the original data was recovered.

“In this demonstration, the scenery of the Odaiba Tokyo Bay area was successfully displayed on the UHD monitor using the ultra-high throughput provided by the 5G network. This technology can be applied to various industries, including tele-health or tele-education.”

Turn phone horizontal to see full image if viewing on mobile:

2. Immersive video

“Scenery was captured by a 180-degree camera equipped with four lenses pointing four different directions installed in the demonstration room, and captured scenery was distributed to smartphones and tablets over the 5G network.

“Four separate cameras were set up to capture the scenery in different directions, and the video images captured by these cameras were stitched together to generate a 180-degree panoramic video image that enabled multiple simultaneous camera views. Then the video image was compressed and distributed to smartphones or tablets in real-time over the 5G network, which gives users a truly realistic user experience.

“Coupled with a 5G network, this technology can be applied to virtual reality (VR) or augmented reality (AR).”

3. Remote control of robotic arm with ultra-low latency

“A robotic arm played an air hockey game against a human in this demonstration. A camera installed on top of the air hockey table detected the puck’s position to calculate its trajectory.

“The calculated result was then forwarded to the robotic arm control server to control the robotic arm. In this demonstration, the robotic arm was able to strike back the puck shot by the human player on various trajectories. This technology can be applied to factory automation, for example.”

4. Remote rendering by GPU server

“Rendering is a technology used to generate videos or images using computers with GPUs (Graphic Processor Unit). This technology is used for generating HD videos in computer games or for CAD (Computer Aided Design). The rendering consumes a large amount of computing resources. Therefore, HD computer games or HD CADs were not executable on tablets or smartphones on their own.

“However, edge computing technology provided by the 5G network allows us to enjoy HD computer games or HD CADs on tablets or smartphones. A GPU server located near a 5G base station performed rendering and the image generated by the GPU server was sent to the tablet over the ultra-high throughput and ultra-low latency 5G network. This technology can be applied to check the CAD data at a construction site with a tablet or to enjoy a HD game application on a smartphone.”

Huawei and Softbank note that: “Immersive video” and “remote control of a robotic arm with ultra-low latency” were jointly integrated and configured for demonstration by SoftBank and Huawei. “UHD real-time video transmission” and “Remote rendering with GPU servers” were integrated and configured for demonstration by SoftBank.

Here are the photos of the Air Hockey game in action:

 

5G use cases demonstrated by SoftBank and Huawei

Source: https://www.itwire.com/telecoms-and-nbn/79837-5g-use-cases-demonstrated-by-softbank-and-huawei.html

5G Rollout In The US: Expected Launch Date, Speeds And Functionality

10 Sep

Super-Fast 5G networks are expected to change the way we use the internet.

AT&T is testing 5G out in the real world in partnership with Intel and Ericsson.
The rollout of 5G networks has been anticipated ever since 4G took off. However, it is yet to become a reality. Yet, it is the need of the hour in the age of smart homes, connected cars, and connected devices.

5G is expected to be a major improvement over 4G and might offer speed of over 1 GB per second. According to the International Telecommunication Union’s 5G standard, 5G networks might offer peak speed of 20 GB per second downlink and 10 GB uplink. The real-world data speed is expected to come up to with at least 100 MB per second.

It is expected to cause an increase in consumer data usage which will make the usage of all things connected whether it is smartphones, smart speakers or cars, much easier for users.

Since it will be around 30-50 times faster than the current data speeds, it will make overall usage of smart devices smoother and easier. It might make for more devices such as the upcoming Apple Watch 3 to become LTE capable i.e. devices would come with embedded SIM cards, providing them data rather than being Wi-Fi dependent.

But 5G has been in the works for long. When will it actually launch?

5G networks are expected to launch by 2020 and according to Gartner, they might cause a three-fold increase in number of connected devices. Whenever it is launched, 5G will support more devices than current 4G ones.

It might also lure consumers into using more value-added services which might make it a more profitable deal for network providers.

Many network providers are already claiming to provide 5G including Verizon and AT&T but the fact remains that none have really stepped up to the mantle by providing actual 1 GB per second speed.

5G will need a strong signal and its signals are high and short, therefore, network provider will have to protect their networks against obstructions.

While network providers might get their act together, most probably by 2020, the hardware will also have to come up to par. Smartphones and other smart devices will have to be equipped with 5G-capable bands.

For example, Apple has already received approval from the Federal Communications Commission for testing 5G broadband and is expected to make its upcoming phones, including the iPhone 8, 5G-capable. Samsung’s Galaxy S8 and Note 8 run on AT&T networks also claim to be 5G-capable.

5G is also expected to accelerate the adoption of technologies such as virtual reality and augmented reality and also increase the presence of more artificial-intelligence based apps and games on connected devices.

That being said, 5G also has risks of exposing users to increased radiation. According to the National Toxicology Program, increased radiation might risk in an increase in the occurrence of tumors.

All such issues will need to be worked out before the commercial deployment of 5G.

Source: http://www.cetusnews.com/tech/5G-Rollout-In-The-US–Expected-Launch-Date–Speeds-And-Functionality.B1ee4N2M9-.html

%d bloggers like this: