Как манипулировать данными в кластере Hadoop?

Question

Как читать и записывать файлы в Hadoop Cluster: Какие операции чтения и записи файлов и команды HDFS для этого?

Answer 1

использование hadoop команда.

hadoop dfs -copyToLocal [-ignoreCrc] [-crc] <src> <localdst> 
this commads write the file to local machine 

hadoop dfs -copyFromLocal <local-file-path> <Hdfs-location> 
this command write file into hdfs 

для получения дополнительной команды просто написать

hadoop dfs -help 

hadoop fs is the command to execute fs commands. The full syntax is: 

hadoop fs [-fs <local | file system URI>] [-conf <configuration file>] 
[-D <property=value>] [-ls <path>] [-lsr <path>] [-du <path>] 
[-dus <path>] [-mv <src> <dst>] [-cp <src> <dst>] [-rm [-skipTrash] <src>] 
[-rmr [-skipTrash] <src>] [-put <localsrc> ... <dst>] [-copyFromLocal <localsrc> ... <dst>] 
[-moveFromLocal <localsrc> ... <dst>] [-get [-ignoreCrc] [-crc] <src> <localdst> 
[-getmerge <src> <localdst> [addnl]] [-cat <src>] 
[-copyToLocal [-ignoreCrc] [-crc] <src> <localdst>] [-moveToLocal <src> <localdst>] 
[-mkdir <path>] [-report] [-setrep [-R] [-w] <rep> <path/file>] 
[-touchz <path>] [-test -[ezd] <path>] [-stat [format] <path>] 
[-tail [-f] <path>] [-text <path>] 
[-chmod [-R] <MODE[,MODE]... | OCTALMODE> PATH...] 
[-chown [-R] [OWNER][:[GROUP]] PATH...] 
[-chgrp [-R] GROUP PATH...] 
[-count[-q] <path>] 
[-help [cmd]]