Join Telegram Study Group ▷
Today's big data has become more and more popular. Today, SPOTO gives you a brief introduction to several representative big data analysis platforms:
1. Cloudera
As the world's most recognized big data platform company, 90% of the domestic version is basically packaged on the basis of CDH and has the greatest control over the community. Partnering with international software vendors, the product covers big data platforms, ETL, advanced analytics, data visualization and more. Cloudera provides a scalable, flexible, and integrated platform for easily managing the rapidly growing variety of data in your enterprise to deploy and manage Hadoop and related projects, operate and analyze your data, and secure your data. Cloudera Manager is a sophisticated application for deploying, managing, monitoring CDH deployments and diagnosing problems. Cloudera Manager provides Admin Console, a web-based user interface that makes your enterprise data management simple and straightforward. Includes the Cloudera Manager API for obtaining cluster health information and metrics and configuring Cloudera Manager.
There is also a professional software such as Statistica, a system environment that integrates data analysis, charting, database management, and custom application development, which not only provides general purpose requirements such as user statistics, mapping, and data management procedures but also provides specific needs. Data analysis method; Actian analysis platform has high availability performance, can be freely deployed in private cloud or hybrid cloud platform, flexible authorization mode, ad hoc query analysis, etc., especially greatly expanding the performance limitations of Hadoop, helping enterprises to convert big data For business value; the Informatics platform is a comprehensive technology that supports multiple complex enterprise-level data integration initiatives, including enterprise data integration, big data, data quality control, master data management, B2B Data Exchange, and application information lifecycle. Management, complex event processing, super messaging, and cloud data integration.
2. Star Ring Transwarp
Based on the Hadoop ecosystem, Big Data Platform, the only big data platform company in the Gartner Magic Quadrant in China, optimized the unstable part of Hadoop, refined its functions, and provided Hadoop big data engine and database tools for enterprises. The underlying is based on a spark, supports SQL on Hadoop, supports sql2003 standard syntax, supports Oracle, DB2, Teradata stored procedures, supports ACID distributed transaction processing, supports efficient memory, SSD computing, and supports visual rights management, computing resource configuration, user security Authorization management, and row-level security controls.
3. Ali number plus
Alibaba Cloud's one-stop big data platform covers the fields of enterprise warehouse, business intelligence, machine learning, data visualization, etc. It can provide data collection, data deep integration, calculation and mining services, and several calculations through visual tools. Personalized data analysis and presentation, graphical display and customer perception are good but need to bundle Alibaba Cloud to use, some experience functions in general, need to have a certain knowledge base. Max compute (formerly known as ODPS) is a multi-level computing engine. There are two dimensions to see the performance of this computing engine. One is to process 100PB of data in 6 hours, which is equivalent to 100 million HD movies, and the other is a single cluster scale of over 10,000. Taiwan, and support multi-cluster joint computing.
The Digital Plus platform consists of three components, development kits, solutions, and data markets. The development kit includes a data development kit and an application development kit. In the data development kit, it mainly includes big data development: integrated visual development environment, which can realize functions of data development, scheduling, deployment, operation and maintenance, data warehouse management, data quality management, etc.; BI reporting tools: real-time online analysis of massive data Rich visual effects; machine learning tools: a machine learning platform that integrates data processing, feature engineering, modeling, and offline prediction. Solution: A number of business scenarios for different business scenarios, based on the development kits provided by the platform and the capabilities of industry service providers, to connect multiple products in series to provide industry solutions.
4. Huawei FusionInsight
A unified platform for enterprise-class big data storage, query, and analysis based on Apache. The fully open big data platform can run on open x86 architecture servers. It is based on a massive data processing engine and real-time data processing engine. It is aimed at the operation and maintenance and application development of data-intensive industries such as finance and operators. Create agile, intelligent, and trusted platform software.
It includes the main software of the open community and its mainstream components in the ecosystem and has been extensively optimized. FusionInsight Stream is a real-time data processing engine in the fusion night big data analysis platform, which solves the real-time data big data technology in the practice-driven mode. The real-time calculation problem of high-speed event flow can realize the real-time processing advantages of streaming events in the fields of finance, communication, transportation, public safety, etc., and provide real-time analysis and real-time decision-making ability.
5. Netease mammoth
Netease Mammoth Big Data Platform is a one-stop big data application development and data management platform, including big data development kit and Hadoop distribution. The big data development kit mainly includes data development, task operation and maintenance, self-service analysis, data management, project management, and multi-tenant management. The Big Data Development Kit effectively links data development, data analysis, data ETL and other data science work through workflows, improving the productivity of data development engineers and data analysis engineers. The Hadoop distribution covers all of the underlying platform components of NetEase Big Data, including self-developed components and components based on open source transformations. Rich and comprehensive components provide comprehensive platform capabilities that make it easy to build solutions in different areas to meet different types of business needs.
The Mammoth platform provides multi-tenant support, and different tenants are isolated from each other. The underlying layer uses Kerberos authentication to achieve data security and isolation. In addition to the authentication system, Granger implements fine-grained access control, ensuring that each tenant can only view the libraries, tables, or fields that are authorized for access. In addition, the platform provides an auditing function to help the post-production of compliance reports and incidents to trace the source and improve platform security through the recording, analysis, and reporting of user platform behavior.
The user operation surface of the platform based on the business scenario design improves the ease of use of the system and ends the cumbersome state of the platform command line operation and maintenance. Data development engineers and data analysts complete data science-related work with simple drag-and-drop and form filling.
The data development module of Big Data Development Kit provides agile development interface for database transfer, SQL, Spark, OLAP Cube, MapReduce and Script various types of tasks. Task developers can create data by drag and drop, and easily integrate data and data ETL. Data science work such as data analysis. Taking database transfer as an example, the user only needs to drag and drop the "database transfer" component onto the canvas and double-click it, and select and manually input the filled form through the drop-down box to quickly complete the task development of data transfer.
In addition, enterprises can perform task scheduling management on demand according to their own business scenarios. Users can set the execution order, priority, and execution period of tasks. Set the number of retries, retry interval, and alarm rules for the failure of the task. Finally, the results of the task can be used to visualize the data analysis of the mainstream BI system, or directly back to the online system to support the auxiliary online business.
In the process of analyzing and processing data, the security importance of data is self-evident. Netease mammoth big data platform uses Kerberos authentication at the bottom, which realizes data security and isolation. In addition to the authentication system, Granger implements fine-grained access control, ensuring that each tenant can only view the tables, libraries, and fields that are authorized for access. Not only that, but the platform also provides an audit function to record, analyze, and report on user platform behavior to help trace the source of the incident and improve the security of the platform.
