MapReduce On HBase

最新推荐文章于 2022-04-17 23:28:33 发布

套码的汉子

最新推荐文章于 2022-04-17 23:28:33 发布

阅读量1k

点赞数

CC 4.0 BY-SA版权

分类专栏： Hbase Java 文章标签： mapreduce

本文链接：https://blog.csdn.net/qq_43502858/article/details/95126283

本文介绍了如何在HBase上使用MapReduce进行数据处理。首先，详细讲述了如何集成HBase与MapReduce，包括从HBase读取数据和将数据写回HBase。接着，提供了一个具体案例，即统计HBase表'mr_people'中不同城市不同年龄段的居民人数，并将结果存入'wc_people'表。案例涵盖了数据准备、MapReduce代码编写及程序运行结果的展示。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

文章目录

MapReduce On MapReduce

MapReduce On MapReduce

HBase与MapReduce集成整合

从HBase里读取数据
- 将HBase里的数据作为Map的输入
将数据写入HBase
- 将HBase作为Reduce的输出
从HBase里读取数据，再写入HBase
- 数据迁移

MapReduce On Hbase小案例

需求：HBase中有一张表’mr_people’,用于存储不同城市的居民信息，现在需要对不同城市不同年龄段的居民人数进行统计，年龄分段为18岁以下，18岁到60岁，60岁以上。并将统计结果存储到HBase的’wc_people’中。

1.准备数据

（1）在HBase中创建一张表’mr_people’用于存储实验数据。

create 'mr_people','person_info'

（2）编写Java代码往表’mr_prople’中存储数据。

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.client.Table;
import org.apache.hadoop.hbase.util.Bytes;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Random;

public class DataGenerator {
   
   
    static Random r = new Random();
    static Configuration conf;
    static String[] citys = new String[]{
   
   "xian","beijing","shanghai","shengzhan","chengdu","chongxing","wuhan","nanchang","hangzhou"};
    static String[] genders = new String[]{
   
   "male","female"};
    static String[] fnames = new String[]{
   
   "Alex","Nike","Mike","Rose","John","Bob","Tony","Nona","Dahl","JoJo","Karol","Handy","Wiki",
    "Aby","Watts","Lamb","Fran","Riggs","Heller","Hank"};
    static String[] lnames = new String[]{
   
   "Smith","Johnson","Williams","Jones","Brown","Davis","Miller","Wilson","Moore","Taylor","Anderson"
    ,"Thomas","Jackson","White","Harris","Jackson","Martin","Garcia"};
    static String[] telHead = new String[]{
   
   "131","135","138","157","181","188","189"};
    static String[] mnames = new String[]{
   
   "D","M","S","J","B","L","N","P","Q","Z","X","Y","C","E"};

    static {
   
   
        conf = new Configuration();
        conf.addResource("hbase-site.xml");
    }

    public static Connection getConn() throws IOException