2014-10-05 5 views
0

У меня есть большой форматированный файл JSON (254 МБ), который я пытаюсь импортировать в MongoDB в Cloud9 IDE с помощью mongoimport. Файл Json состоит из документов mongodb каждый в своей строке с использованием символа новой строки. Файл выглядит следующим образом:MongoDB mongoimport дает ошибку при импорте большого json-файла

... 
{"t":"1358836264","p":"1.33470"} 
{"t":"1358836265","p":"1.33475"} 
{"t":"1358836271","p":"1.33477"} 
{"t":"1358836272","p":"1.33481"} 
{"t":"1358836274","p":"1.33475"} 
{"t":"1358836275","p":"1.33478"} 
{"t":"1358836288","p":"1.33480"} 
{"t":"1358836291","p":"1.33481"} 
{"t":"1358836294","p":"1.33481"} 
{"t":"1358836295","p":"1.33478"} 
... 

Я попытался:

mongoimport -d woption -c eurusd eurusddata.json 

Это дало мне следующий результат с ошибкой:

connected to: 127.0.0.1 
Sun Oct 5 16:51:15.094     Progress: 1636800/26345472  6% 
Sun Oct 5 16:51:15.094       49600 16533/second 
Sun Oct 5 16:51:18.088     Progress: 3375900/26345472  12% 
Sun Oct 5 16:51:18.088       102300 17050/second 
Sun Oct 5 16:51:21.089     Progress: 4867500/26345472  18% 
Sun Oct 5 16:51:21.089       147500 16388/second 
Sun Oct 5 16:51:24.103     Progress: 7728600/26345472  29% 
Sun Oct 5 16:51:24.103       234200 19516/second 
Sun Oct 5 16:51:27.093     Progress: 10467600/26345472  39% 
Sun Oct 5 16:51:27.093       317200 21146/second 
Sun Oct 5 16:51:30.094     Progress: 13312200/26345472  50% 
Sun Oct 5 16:51:30.094       403400 22411/second 
Sun Oct 5 16:51:33.302     Progress: 16038000/26345472  60% 
Sun Oct 5 16:51:33.303       486000 23142/second 
Sun Oct 5 16:51:36.088     Progress: 17341500/26345472  65% 
Sun Oct 5 16:51:36.088       525500 21895/second 
Sun Oct 5 16:51:39.004     Progress: 18526200/26345472  70% 
Sun Oct 5 16:51:39.004       561400 20792/second 
Sun Oct 5 16:51:42.032     Progress: 19067400/26345472  72% 
Sun Oct 5 16:51:42.032       577800 19260/second 
Sun Oct 5 16:51:45.088     Progress: 20829600/26345472  79% 
Sun Oct 5 16:51:45.088       631200 19127/second 
Sun Oct 5 16:51:48.071     Progress: 23007600/26345472  87% 
Sun Oct 5 16:51:48.071       697200 19366/second 
Sun Oct 5 16:51:51.914     Progress: 23443200/26345472  88% 
Sun Oct 5 16:51:51.914       710400 18215/second 
Sun Oct 5 16:51:54.103     Progress: 23611500/26345472  89% 
Sun Oct 5 16:51:54.104       715500 17035/second 
Sun Oct 5 16:51:57.709     Progress: 23967900/26345472  90% 
Sun Oct 5 16:51:57.709       726300 16140/second 
Sun Oct 5 16:52:00.096     Progress: 24538800/26345472  93% 
Sun Oct 5 16:52:00.096       743600 15491/second 
Sun Oct 5 16:52:03.088     Progress: 25548600/26345472  96% 
Sun Oct 5 16:52:03.088       774200 15180/second 
Sun Oct 5 16:52:04.644 exception:BSON representation of supplied JSON is too large: code FailedToParse: FailedToParse: Expecting ':': offset:21 
Sun Oct 5 16:52:04.644 
Sun Oct 5 16:52:04.644 check 9 798347 
Sun Oct 5 16:52:05.061 imported 798347 objects 
Sun Oct 5 16:52:05.061 ERROR: encountered 1 error(s) 

Затем я попытался с помощью --jsonArray в конце от предыдущей команды:

mongoimport -d woption -c eurusd eurusddata.json --jsonArray 

Это дало мне следующий результат с ошибкой:

connected to: 127.0.0.1 
Sun Oct 5 16:54:20.343 exception:JSONArray file too large 
Sun Oct 5 16:54:20.444 warning: log line attempted (16384k) over max size(10k), printing beginning and end ... {"t":"1357070402","p":"1.32041"} 
{"t":"1357070424","p":"1.32040"} 
{"t":"1357070447","p":"1.32038"} 
{"t":"1357070457","p":"1.32034"} 
{"t":"1357070463","p":"1.32039"} 
{"t":"1357070464","p":"1.32038"} 
{"t":"1357070470","p":"1.32034"} 
{"t":"1357070485","p":"1.32035"} 
{"t":"1357070491","p":"1.32046"} 
{"t":"1357070520","p":"1.32050"} 
{"t":"1357070522","p":"1.32049"} 
{"t":"1357070535","p":"1.32050"} 
{"t":"1357070581","p":"1.32049"} 
{"t":"1357070582","p":"1.32050"} 
{"t":"1357070587","p":"1.32035"} 
{"t":"1357070589","p":"1.32034"} 
{"t":"1357070593","p":"1.32033"} 
{"t":"1357070594","p":"1.32024"} 
{"t":"1357070595","p":"1.32025"} 
{"t":"1357070599","p":"1.32009"} 
{"t":"1357070602","p":"1.32024"} 
{"t":"1357070630","p":"1.32023"} 
{"t":"1357070637","p":"1.32025"} 
{"t":"1357070656","p":"1.32023"} 
{"t":"1357070691","p":"1.32025"} 
{"t":"1357070702","p":"1.32027"} 
{"t":"1357070703","p":"1.32028"} 
{"t":"1357070706","p":"1.32027"} 
{"t":"1357070707","p":"1.32026"} 
{"t":"1357070709","p":"1.32027"} 
{"t":"1357070710","p":"1.32026"} 
{"t":"1357070721","p":"1.32027"} 
{"t":"1357070723","p":"1.32026"} 
{"t":"1357070761","p":"1.32025"} 
{"t":"1357070767","p":"1.32027"} 
{"t":"1357070768","p":"1.32025"} 
{"t":"1357070784","p":"1.32026"} 
{"t":"1357070798","p":"1.32027"} 
{"t":"1357070799","p":"1.32026"} 
{"t":"1357070804","p":"1.32036"} 
{"t":"1357070819","p":"1.32034"} 
{"t":"1357070842","p":"1.32035"} 
{"t":"1357070880","p":"1.32030"} 
{"t":"1357070881","p":"1.32035"} 
{"t":"1357070966","p":"1.32053"} 
{"t":"1357070967","p":"1.32063"} 
{"t":"1357070973","p":"1.32048"} 
{"t":"1357070974","p":"1.32046"} 
{"t":"1357070977","p":"1.32066"} 
{"t":"1357070978","p":"1.32065"} 
{"t":"1357070984","p":"1.32064"} 
{"t":"1357070987","p":"1.32063"} 
{"t":"1357070988","p":"1.32064"} 
{"t":"1357070993","p":"1.32065"} 
{"t":"1357070997","p":"1.32065"} 
{"t":"1357070998","p":"1.32063"} 
{"t":"1357071000","p":"1.32064"} 
{"t":"1357071003","p":"1.32064"} 
{"t":"1357071011","p":"1.32065"} 
{"t":"1357071012","p":"1.32069"} 
{"t":"1357071013","p":"1.32069"} 
{"t":"1357071023","p":"1.32068"} 
{"t":"1357071024","p":"1.32068"} 
{"t":"1357071026","p":"1.32072"} 
{"t":"1357071027","p":"1.32070"} 
{"t":"1357071028","p":"1.32069"} 
{"t":"1357071047","p":"1.32068"} 
{"t":"1357071048","p":"1.32069"} 
{"t":"1357071070","p":"1.32072"} 
{"t":"1357071071","p":"1.32069"} 
{"t":"1357071077","p":"1.32068"} 
{"t":"1357071078","p":"1.32069"} 
{"t":"1357071085","p":"1.32068"} 
{"t":"1357071086","p":"1.32069"} 
{"t":"1357071095","p":"1.32072"} 
{"t":"1357071096","p":"1.32069"} 
{"t":"1357071097","p":"1.32070"} 
{"t":"1357071104","p":"1.32069"} 
{"t":"1357071105","p":"1.32069"} 
{"t":"1357071114","p":"1.32068"} 
{"t":"1357071125","p":"1.32068"} 
{"t":"1357071132","p":"1.32072"} 
{"t":"1357071133","p":"1.32068"} 
{"t":"1357071134","p":"1.32069"} 
{"t":"1357071136","p":"1.32043"} 
{"t":"1357071137","p":"1.32049"} 
{"t":"1357071139","p":"1.32052"} 
{"t":"1357071140","p":"1.32058"} 
{"t":"1357071156","p":"1.32052"} 
{"t":"1357071163","p":"1.32050"} 
{"t":"1357071174","p":"1.32052"} 
{"t":"1357071176","p":"1.32055"} 
{"t":"1357071178","p":"1.32053"} 
{"t":"1357071185","p":"1.32054"} 
{"t":"1357071187","p":"1.32053"} 
{"t":"1357071189","p":"1.32025"} 
{"t":"1357071190","p":"1.32023"} 
{"t":"1357071191","p":"1.32045"} 
{"t":"1357071192","p":"1.32044"} 
{"t":"1357071193","p":"1.32045"} 
{"t":"1357071197","p":"1.32044"} 
{"t":"1357071198","p":"1.32043"} 
{"t":"1357071199","p":"1.32004"} 
{"t":"13570712 .......... ":"1358836127","p":"1.33455"} 
{"t":"1358836128","p":"1.33454"} 
{"t":"1358836129","p":"1.33455"} 
{"t":"1358836130","p":"1.33458"} 
{"t":"1358836131","p":"1.33463"} 
{"t":"1358836132","p":"1.33462"} 
{"t":"1358836133","p":"1.33460"} 
{"t":"1358836134","p":"1.33459"} 
{"t":"1358836135","p":"1.33464"} 
{"t":"1358836137","p":"1.33463"} 
{"t":"1358836139","p":"1.33461"} 
{"t":"1358836140","p":"1.33462"} 
{"t":"1358836142","p":"1.33463"} 
{"t":"1358836143","p":"1.33459"} 
{"t":"1358836144","p":"1.33461"} 
{"t":"1358836146","p":"1.33462"} 
{"t":"1358836147","p":"1.33466"} 
{"t":"1358836148","p":"1.33473"} 
{"t":"1358836149","p":"1.33475"} 
{"t":"1358836151","p":"1.33479"} 
{"t":"1358836152","p":"1.33484"} 
{"t":"1358836153","p":"1.33492"} 
{"t":"1358836154","p":"1.33488"} 
{"t":"1358836155","p":"1.33492"} 
{"t":"1358836156","p":"1.33489"} 
{"t":"1358836157","p":"1.33482"} 
{"t":"1358836158","p":"1.33483"} 
{"t":"1358836159","p":"1.33480"} 
{"t":"1358836160","p":"1.33479"} 
{"t":"1358836161","p":"1.33482"} 
{"t":"1358836162","p":"1.33481"} 
{"t":"1358836163","p":"1.33481"} 
{"t":"1358836166","p":"1.33480"} 
{"t":"1358836167","p":"1.33480"} 
{"t":"1358836168","p":"1.33475"} 
{"t":"1358836169","p":"1.33454"} 
{"t":"1358836170","p":"1.33454"} 
{"t":"1358836172","p":"1.33459"} 
{"t":"1358836173","p":"1.33456"} 
{"t":"1358836174","p":"1.33455"} 
{"t":"1358836175","p":"1.33457"} 
{"t":"1358836176","p":"1.33459"} 
{"t":"1358836177","p":"1.33460"} 
{"t":"1358836178","p":"1.33462"} 
{"t":"1358836179","p":"1.33458"} 
{"t":"1358836180","p":"1.33459"} 
{"t":"1358836181","p":"1.33456"} 
{"t":"1358836182","p":"1.33458"} 
{"t":"1358836183","p":"1.33457"} 
{"t":"1358836184","p":"1.33459"} 
{"t":"1358836185","p":"1.33460"} 
{"t":"1358836186","p":"1.33467"} 
{"t":"1358836189","p":"1.33466"} 
{"t":"1358836190","p":"1.33472"} 
{"t":"1358836191","p":"1.33470"} 
{"t":"1358836192","p":"1.33470"} 
{"t":"1358836193","p":"1.33469"} 
{"t":"1358836194","p":"1.33465"} 
{"t":"1358836195","p":"1.33466"} 
{"t":"1358836196","p":"1.33461"} 
{"t":"1358836197","p":"1.33460"} 
{"t":"1358836198","p":"1.33466"} 
{"t":"1358836199","p":"1.33465"} 
{"t":"1358836200","p":"1.33466"} 
{"t":"1358836201","p":"1.33458"} 
{"t":"1358836202","p":"1.33457"} 
{"t":"1358836203","p":"1.33453"} 
{"t":"1358836204","p":"1.33454"} 
{"t":"1358836205","p":"1.33456"} 
{"t":"1358836210","p":"1.33455"} 
{"t":"1358836211","p":"1.33450"} 
{"t":"1358836214","p":"1.33449"} 
{"t":"1358836216","p":"1.33451"} 
{"t":"1358836217","p":"1.33452"} 
{"t":"1358836218","p":"1.33457"} 
{"t":"1358836220","p":"1.33458"} 
{"t":"1358836221","p":"1.33457"} 
{"t":"1358836222","p":"1.33456"} 
{"t":"1358836223","p":"1.33457"} 
{"t":"1358836226","p":"1.33460"} 
{"t":"1358836229","p":"1.33459"} 
{"t":"1358836230","p":"1.33458"} 
{"t":"1358836236","p":"1.33461"} 
{"t":"1358836237","p":"1.33466"} 
{"t":"1358836240","p":"1.33467"} 
{"t":"1358836241","p":"1.33467"} 
{"t":"1358836242","p":"1.33463"} 
{"t":"1358836243","p":"1.33466"} 
{"t":"1358836244","p":"1.33467"} 
{"t":"1358836245","p":"1.33468"} 
{"t":"1358836250","p":"1.33468"} 
{"t":"1358836256","p":"1.33469"} 
{"t":"1358836263","p":"1.33472"} 
{"t":"1358836264","p":"1.33470"} 
{"t":"1358836265","p":"1.33475"} 
{"t":"1358836271","p":"1.33477"} 
{"t":"1358836272","p":"1.33481"} 
{"t":"1358836274","p":"1.33475"} 
{"t":"1358836275","p":"1.33478"} 
{"t":"1358836288","p":"1.33480"} 
{"t":"1358836291","p":"1.33481"} 
{"t":"1358836294","p":"1.33481"} 
{"t":"1358836295","p":"1.33478"} 
{"t":"1358836296 
Sun Oct 5 16:54:20.445 check 0 0 
Sun Oct 5 16:54:20.445 imported 0 objects 
Sun Oct 5 16:54:20.445 ERROR: encountered 1 error(s) 

Затем я попытался использовать что-то вроде этого:

mongoimport -d woption -c eurusd < /home/ubuntu/workspace/eurusddata.json 

Это дало мне следующий результат с ошибкой:

connected to: 127.0.0.1 
Sun Oct 5 16:58:08.107       77400 25800/second 
Sun Oct 5 16:58:11.096       136300 22716/second 
Sun Oct 5 16:58:14.107       177700 19744/second 
Sun Oct 5 16:58:17.093       232900 19408/second 
Sun Oct 5 16:58:20.125       280400 18693/second 
Sun Oct 5 16:58:23.103       343500 19083/second 
Sun Oct 5 16:58:26.091       424800 20228/second 
Sun Oct 5 16:58:29.095       482800 20116/second 
Sun Oct 5 16:58:32.097       517900 19181/second 
Sun Oct 5 16:58:35.088       566700 18890/second 
Sun Oct 5 16:58:38.091       608700 18445/second 
Sun Oct 5 16:58:41.098       664700 18463/second 
Sun Oct 5 16:58:44.088       718300 18417/second 
Sun Oct 5 16:58:47.089       776800 18495/second 
Sun Oct 5 16:58:47.848 exception:BSON representation of supplied JSON is too large: code FailedToParse: FailedToParse: Expecting ':': offset:21 
Sun Oct 5 16:58:47.848 
Sun Oct 5 16:58:47.848 check 9 798347 
Sun Oct 5 16:58:48.852 imported 798347 objects 
Sun Oct 5 16:58:48.852 ERROR: encountered 1 error(s) 

пытался также используя предыдущую команду с --jsonArray:

mongoimport -d woption -c eurusd < /home/ubuntu/workspace/eurusddata.json --jsonArray 

Это дало мне следующий результат с ошибкой:

connected to: 127.0.0.1 
Sun Oct 5 16:59:35.063 exception:JSONArray file too large 
Sun Oct 5 16:59:35.101 warning: log line attempted (16384k) over max size(10k), printing beginning and end ... {"t":"1357070402","p":"1.32041"} 
{"t":"1357070424","p":"1.32040"} 
{"t":"1357070447","p":"1.32038"} 
{"t":"1357070457","p":"1.32034"} 
{"t":"1357070463","p":"1.32039"} 
{"t":"1357070464","p":"1.32038"} 
{"t":"1357070470","p":"1.32034"} 
{"t":"1357070485","p":"1.32035"} 
{"t":"1357070491","p":"1.32046"} 
{"t":"1357070520","p":"1.32050"} 
{"t":"1357070522","p":"1.32049"} 
{"t":"1357070535","p":"1.32050"} 
{"t":"1357070581","p":"1.32049"} 
{"t":"1357070582","p":"1.32050"} 
{"t":"1357070587","p":"1.32035"} 
{"t":"1357070589","p":"1.32034"} 
{"t":"1357070593","p":"1.32033"} 
{"t":"1357070594","p":"1.32024"} 
{"t":"1357070595","p":"1.32025"} 
{"t":"1357070599","p":"1.32009"} 
{"t":"1357070602","p":"1.32024"} 
{"t":"1357070630","p":"1.32023"} 
{"t":"1357070637","p":"1.32025"} 
{"t":"1357070656","p":"1.32023"} 
{"t":"1357070691","p":"1.32025"} 
{"t":"1357070702","p":"1.32027"} 
{"t":"1357070703","p":"1.32028"} 
{"t":"1357070706","p":"1.32027"} 
{"t":"1357070707","p":"1.32026"} 
{"t":"1357070709","p":"1.32027"} 
{"t":"1357070710","p":"1.32026"} 
{"t":"1357070721","p":"1.32027"} 
{"t":"1357070723","p":"1.32026"} 
{"t":"1357070761","p":"1.32025"} 
{"t":"1357070767","p":"1.32027"} 
{"t":"1357070768","p":"1.32025"} 
{"t":"1357070784","p":"1.32026"} 
{"t":"1357070798","p":"1.32027"} 
{"t":"1357070799","p":"1.32026"} 
{"t":"1357070804","p":"1.32036"} 
{"t":"1357070819","p":"1.32034"} 
{"t":"1357070842","p":"1.32035"} 
{"t":"1357070880","p":"1.32030"} 
{"t":"1357070881","p":"1.32035"} 
{"t":"1357070966","p":"1.32053"} 
{"t":"1357070967","p":"1.32063"} 
{"t":"1357070973","p":"1.32048"} 
{"t":"1357070974","p":"1.32046"} 
{"t":"1357070977","p":"1.32066"} 
{"t":"1357070978","p":"1.32065"} 
{"t":"1357070984","p":"1.32064"} 
{"t":"1357070987","p":"1.32063"} 
{"t":"1357070988","p":"1.32064"} 
{"t":"1357070993","p":"1.32065"} 
{"t":"1357070997","p":"1.32065"} 
{"t":"1357070998","p":"1.32063"} 
{"t":"1357071000","p":"1.32064"} 
{"t":"1357071003","p":"1.32064"} 
{"t":"1357071011","p":"1.32065"} 
{"t":"1357071012","p":"1.32069"} 
{"t":"1357071013","p":"1.32069"} 
{"t":"1357071023","p":"1.32068"} 
{"t":"1357071024","p":"1.32068"} 
{"t":"1357071026","p":"1.32072"} 
{"t":"1357071027","p":"1.32070"} 
{"t":"1357071028","p":"1.32069"} 
{"t":"1357071047","p":"1.32068"} 
{"t":"1357071048","p":"1.32069"} 
{"t":"1357071070","p":"1.32072"} 
{"t":"1357071071","p":"1.32069"} 
{"t":"1357071077","p":"1.32068"} 
{"t":"1357071078","p":"1.32069"} 
{"t":"1357071085","p":"1.32068"} 
{"t":"1357071086","p":"1.32069"} 
{"t":"1357071095","p":"1.32072"} 
{"t":"1357071096","p":"1.32069"} 
{"t":"1357071097","p":"1.32070"} 
{"t":"1357071104","p":"1.32069"} 
{"t":"1357071105","p":"1.32069"} 
{"t":"1357071114","p":"1.32068"} 
{"t":"1357071125","p":"1.32068"} 
{"t":"1357071132","p":"1.32072"} 
{"t":"1357071133","p":"1.32068"} 
{"t":"1357071134","p":"1.32069"} 
{"t":"1357071136","p":"1.32043"} 
{"t":"1357071137","p":"1.32049"} 
{"t":"1357071139","p":"1.32052"} 
{"t":"1357071140","p":"1.32058"} 
{"t":"1357071156","p":"1.32052"} 
{"t":"1357071163","p":"1.32050"} 
{"t":"1357071174","p":"1.32052"} 
{"t":"1357071176","p":"1.32055"} 
{"t":"1357071178","p":"1.32053"} 
{"t":"1357071185","p":"1.32054"} 
{"t":"1357071187","p":"1.32053"} 
{"t":"1357071189","p":"1.32025"} 
{"t":"1357071190","p":"1.32023"} 
{"t":"1357071191","p":"1.32045"} 
{"t":"1357071192","p":"1.32044"} 
{"t":"1357071193","p":"1.32045"} 
{"t":"1357071197","p":"1.32044"} 
{"t":"1357071198","p":"1.32043"} 
{"t":"1357071199","p":"1.32004"} 
{"t":"13570712 .......... ":"1358836127","p":"1.33455"} 
{"t":"1358836128","p":"1.33454"} 
{"t":"1358836129","p":"1.33455"} 
{"t":"1358836130","p":"1.33458"} 
{"t":"1358836131","p":"1.33463"} 
{"t":"1358836132","p":"1.33462"} 
{"t":"1358836133","p":"1.33460"} 
{"t":"1358836134","p":"1.33459"} 
{"t":"1358836135","p":"1.33464"} 
{"t":"1358836137","p":"1.33463"} 
{"t":"1358836139","p":"1.33461"} 
{"t":"1358836140","p":"1.33462"} 
{"t":"1358836142","p":"1.33463"} 
{"t":"1358836143","p":"1.33459"} 
{"t":"1358836144","p":"1.33461"} 
{"t":"1358836146","p":"1.33462"} 
{"t":"1358836147","p":"1.33466"} 
{"t":"1358836148","p":"1.33473"} 
{"t":"1358836149","p":"1.33475"} 
{"t":"1358836151","p":"1.33479"} 
{"t":"1358836152","p":"1.33484"} 
{"t":"1358836153","p":"1.33492"} 
{"t":"1358836154","p":"1.33488"} 
{"t":"1358836155","p":"1.33492"} 
{"t":"1358836156","p":"1.33489"} 
{"t":"1358836157","p":"1.33482"} 
{"t":"1358836158","p":"1.33483"} 
{"t":"1358836159","p":"1.33480"} 
{"t":"1358836160","p":"1.33479"} 
{"t":"1358836161","p":"1.33482"} 
{"t":"1358836162","p":"1.33481"} 
{"t":"1358836163","p":"1.33481"} 
{"t":"1358836166","p":"1.33480"} 
{"t":"1358836167","p":"1.33480"} 
{"t":"1358836168","p":"1.33475"} 
{"t":"1358836169","p":"1.33454"} 
{"t":"1358836170","p":"1.33454"} 
{"t":"1358836172","p":"1.33459"} 
{"t":"1358836173","p":"1.33456"} 
{"t":"1358836174","p":"1.33455"} 
{"t":"1358836175","p":"1.33457"} 
{"t":"1358836176","p":"1.33459"} 
{"t":"1358836177","p":"1.33460"} 
{"t":"1358836178","p":"1.33462"} 
{"t":"1358836179","p":"1.33458"} 
{"t":"1358836180","p":"1.33459"} 
{"t":"1358836181","p":"1.33456"} 
{"t":"1358836182","p":"1.33458"} 
{"t":"1358836183","p":"1.33457"} 
{"t":"1358836184","p":"1.33459"} 
{"t":"1358836185","p":"1.33460"} 
{"t":"1358836186","p":"1.33467"} 
{"t":"1358836189","p":"1.33466"} 
{"t":"1358836190","p":"1.33472"} 
{"t":"1358836191","p":"1.33470"} 
{"t":"1358836192","p":"1.33470"} 
{"t":"1358836193","p":"1.33469"} 
{"t":"1358836194","p":"1.33465"} 
{"t":"1358836195","p":"1.33466"} 
{"t":"1358836196","p":"1.33461"} 
{"t":"1358836197","p":"1.33460"} 
{"t":"1358836198","p":"1.33466"} 
{"t":"1358836199","p":"1.33465"} 
{"t":"1358836200","p":"1.33466"} 
{"t":"1358836201","p":"1.33458"} 
{"t":"1358836202","p":"1.33457"} 
{"t":"1358836203","p":"1.33453"} 
{"t":"1358836204","p":"1.33454"} 
{"t":"1358836205","p":"1.33456"} 
{"t":"1358836210","p":"1.33455"} 
{"t":"1358836211","p":"1.33450"} 
{"t":"1358836214","p":"1.33449"} 
{"t":"1358836216","p":"1.33451"} 
{"t":"1358836217","p":"1.33452"} 
{"t":"1358836218","p":"1.33457"} 
{"t":"1358836220","p":"1.33458"} 
{"t":"1358836221","p":"1.33457"} 
{"t":"1358836222","p":"1.33456"} 
{"t":"1358836223","p":"1.33457"} 
{"t":"1358836226","p":"1.33460"} 
{"t":"1358836229","p":"1.33459"} 
{"t":"1358836230","p":"1.33458"} 
{"t":"1358836236","p":"1.33461"} 
{"t":"1358836237","p":"1.33466"} 
{"t":"1358836240","p":"1.33467"} 
{"t":"1358836241","p":"1.33467"} 
{"t":"1358836242","p":"1.33463"} 
{"t":"1358836243","p":"1.33466"} 
{"t":"1358836244","p":"1.33467"} 
{"t":"1358836245","p":"1.33468"} 
{"t":"1358836250","p":"1.33468"} 
{"t":"1358836256","p":"1.33469"} 
{"t":"1358836263","p":"1.33472"} 
{"t":"1358836264","p":"1.33470"} 
{"t":"1358836265","p":"1.33475"} 
{"t":"1358836271","p":"1.33477"} 
{"t":"1358836272","p":"1.33481"} 
{"t":"1358836274","p":"1.33475"} 
{"t":"1358836275","p":"1.33478"} 
{"t":"1358836288","p":"1.33480"} 
{"t":"1358836291","p":"1.33481"} 
{"t":"1358836294","p":"1.33481"} 
{"t":"1358836295","p":"1.33478"} 
{"t":"1358836296 
Sun Oct 5 16:59:35.101 check 0 0 
Sun Oct 5 16:59:35.102 imported 0 objects 
Sun Oct 5 16:59:35.102 ERROR: encountered 1 error(s) 

Что я делаю неправильно или что я здесь отсутствует? Как импортировать большие json-файлы в MongoDB?

ответ

0

Измените свою команду с mongoimport -d woption -c eurusd eurusddata.json --jsonArray на mongoimport -d woption -c eurusd --file eurusddata.json --jsonArray, и это сработает. Надеюсь это поможет. Обновите, если это не сработает.

+0

Спасибо за помощь. Пробовал вашу предложенную команду - по-прежнему дает ту же ошибку: подключен к: 127.0.0.1 вс 5 окт 19: 01: 08.253 исключение: файл JSONArray слишком большой вс 5 окт 19: 01: 08.293 предупреждение: попытка строки журнала (16384k) (10k), начало и конец печати ... Вс Окт 5 19: 01: 08.294 check 0 0 Sun Oct 5 19:01:08.295 импортированных 0 объектов Вс Окт 05 19: 01: 08.295 ОШИБКА: столкнулись 1 ошибка (ов) – Robert

2

Согласно сообщению:

Sun Oct 5 16:52:03.088     Progress: 25548600/26345472  96% 
Sun Oct 5 16:52:03.088       774200 15180/second 
Sun Oct 5 16:52:04.644 exception:BSON representation of supplied JSON is too large: code FailedToParse: FailedToParse: Expecting ':': offset:21 

Я думаю, что есть какая-то ошибка ближе к концу импортированного файла.

Для {"t":"1358836291","p":"1.33481"}, смещение второго : равно 21. Таким образом, пожалуйста, проверьте синтаксис документов в конце (96% ~ 100%) импортированного файла.
Кстати, --jsonArray не подходит для использования здесь, потому что у него есть предел 16 МБ.

0

Попробуйте этот вариант: --batchSize 1

, как:

mongoimport -d woption -c eurusd eurusddata.json --batchSize 1 
Смежные вопросы