An Optimized Model for MapReduce Based on Hadoop
Abstract: Aiming at the waste
of computing resources resulting from sequential control of running mechanism
of MapReduce model on Hadoop platform，Fork/Join framework has been introduced
into this model to make full use of CPU resource of each node. From the
perspective of fine-grained parallel data processing, combined with Fork/Join
framework，a parallel and multi-thread model，this paper optimizes MapReduce
model and puts forward a MapReduce+Fork/Join programming model which is a
distributed and parallel architecture combined with coarse-grained and
fine-grained on Hadoop platform to Support two-tier levels of parallelism
architecture both in shared and distributed memory machines. A test is made under
the environment of Hadoop cluster composed of four nodes. And the experimental
results prove that this model really can improve performance and efficiency of
the whole system and it is not only suitable for handling tasks with data
intensive but also tasks with computing intensive. it is an effective
optimization and improvement to the MapReduce model of big data processing.
Author: Zhang Hong
Journal Code: jptkomputergg160305
Artikel Terkait :
Jp Teknik Komputer gg 2016
- An Improved Artificial Bee Colony Algorithm for Staged Search
- Multi-Criteria in Discriminant Analysis to Find the Dominant Features
- A Novel Multifunction Digital Chip Design Based on CMOS Technology
- An Improved Adaptive Niche Differential Evolution Algorithm
- Power Quality Analysis of Integration Photovoltaic Generator to Three Phase Grid under Variable Solar Irradiance Level
- A Combined User-order and Chunk-order Algorithm to Minimize the Average BER for Chunk Allocation in SCFDMA Systems
- Hierarchical i* Modeling in Requirement Engineering
- The Optimal High Performance Computing Infrastructure for Solving High Complexity Problem
- Transformer Fault Diagnosis Method Based on Dynamic Weighted Combination Model
- Hybrid Hierarchical Collision Detection Based on Data Reuse
- Brightness and Contrast Modification in Ultrasonography Images Using Edge Detection Results
- Recognition of Fission Signals Based on Wavelet Analysis and Neural Network
- Application of Nonlinear Dynamical Methods for Arc Welding Quality Monitoring
- Chaos-Enhanced Cuckoo Search for Economic Dispatch with Valve Point Effects
- GPU CUDA Accelerated Image Inpainting using Fourth Order PDE Equation
- Comparative Analysis of Spatial Decision Tree Algorithms for Burned Area of Peatland in Rokan Hilir Riau
- Action Recognition of Human’s Lower Limbs Based on a Human Joint
- A Soft Error Study on Tri-gate Based FinFET and Junctionless-FinFET 6T SRAM Cell - A Comparison
- Design of AC Charging Interface and Status Acquisition Circuit for Electric Vehicles
- Towards Smooth and High-Quality Bitrate Adaptation for HTTP Adaptive Streaming
- Features Deletion on Multiple Objects Recognition
- Research on Batch Scheduling in Cloud Computing
- Classification of Motorcyclists not Wear Helmet on Digital Image with Backpropagation Neural Network
- Internet Protocol Based Satellite On-Board System