Parsing Python Запрос JSON вызывает ValueError

Question

Я пытаюсь загрузить строку JSON, удаленную удаленно, используя модуль requests. Содержимое успешно извлекается, но когда я пытаюсь загрузить его в dict, я получаю эту ошибку: ValueError: No JSON object could be decoded.

# -*- coding: utf-8 -*- 
import requests, json 

url = 'some-url.com' 
data = {} # dict with actual content 
headers = {'content-type': 'application/json', 'accept': 'application/json'} 

r = requests.post(url, data=json.dumps(data), headers=headers) 

print r.json() # this line causes the error 

Для справки, вот заголовки ответа:

{'content-length': '14411', 'x-powered-by': 'ASP.NET', 'server': 'Microsoft-IIS/7.5', 'cache-control': 'private', 'date': 'Thu, 19 Jun 2014 00:37:36 GMT', 'access-control-allow-origin': '*', 'content-type': 'application/json; charset=utf-8'} 

EDIT: Содержание r.text

{"columns":[{"code":"Region","text":"region","type":"d"},{"code":"Tid","text":"år","type":"t"},{"code":"BE0101N1","text":"Folkmängd","type":"c"}],"comments":[{"variable":"Region","value":"03","comment":"Fr o m 2007-01-01 utökas Uppsala län med Heby kommun. Observera att länssiffrorna inte är jämförbara med länssiffrorna bakåt i tiden.\r\n"},{"variable":"Region","value":"0330","comment":"Ny regional indelning fr.o.m. 2003-01-01. Delar av Uppsala kommun bildar en ny kommun benämnd Knivsta kommun.\r\n"},{"variable":"Region","value":"0331","comment":"Fr o m 2007-01-01 överförs Heby kommun från Västmanlands län till Uppsala län. Hebys kommunkod ändras från 1917 till 0331.\r\n"},{"variable":"Region","value":"0380","comment":"Ny regional indelning fr.o.m. 2003-01-01. Delar av Uppsala kommun bildar en ny kommun benämnd Knivsta kommun.\r\n"},{"variable":"Region","value":"19","comment":"Fr o m 2007-01-01 minskar Västmanlands län med Heby kommun. Observera att länssiffrorna inte är jämförbara med länssiffrorna bakåt i tiden.\r\n"}],"data":[{"key":["00","2013"],"values":["9644864"]},{"key":["01","2013"],"values":["2163042"]},{"key":["0114","2013"],"values":["41449"]},{"key":["0115","2013"],"values":["31616"]},{"key":["0117","2013"],"values":["40495"]},{"key":["0120","2013"],"values":["39784"]},{"key":["0123","2013"],"values":["69167"]},{"key":["0125","2013"],"values":["26355"]},{"key":["0126","2013"],"values":["102557"]},{"key":["0127","2013"],"values":["87580"]},{"key":["0128","2013"],"values":["16001"]},{"key":["0136","2013"],"values":["80932"]},{"key":["0138","2013"],"values":["44281"]},{"key":["0139","2013"],"values":["24703"]},{"key":["0140","2013"],"values":["9523"]},{"key":["0160","2013"],"values":["66292"]},{"key":["0162","2013"],"values":["32222"]},{"key":["0163","2013"],"values":["68145"]},{"key":["0180","2013"],"values":["897700"]},{"key":["0181","2013"],"values":["91072"]},{"key":["0182","2013"],"values":["94423"]},{"key":["0183","2013"],"values":["42626"]},{"key":["0184","2013"],"values":["72740"]},{"key":["0186","2013"],"values":["45178"]},{"key":["0187","2013"],"values":["11188"]},{"key":["0188","2013"],"values":["56845"]},{"key":["0191","2013"],"values":["43372"]},{"key":["0192","2013"],"values":["26796"]},{"key":["03","2013"],"values":["345481"]},{"key":["0305","2013"],"values":["19968"]},{"key":["0319","2013"],"values":["9132"]},{"key":["0330","2013"],"values":["15580"]},{"key":["0331","2013"],"values":["13450"]},{"key":["0360","2013"],"values":["20144"]},{"key":["0380","2013"],"values":["205199"]},{"key":["0381","2013"],"values":["40656"]},{"key":["0382","2013"],"values":["21352"]},{"key":["04","2013"],"values":["277569"]},{"key":["0428","2013"],"values":["8835"]},{"key":["0461","2013"],"values":["10409"]},{"key":["0480","2013"],"values":["53038"]},{"key":["0481","2013"],"values":["11403"]},{"key":["0482","2013"],"values":["16156"]},{"key":["0483","2013"],"values":["32930"]},{"key":["0484","2013"],"values":["99729"]},{"key":["0486","2013"],"values":["33389"]},{"key":["0488","2013"],"values":["11680"]},{"key":["05","2013"],"values":["437848"]},{"key":["0509","2013"],"values":["5174"]},{"key":["0512","2013"],"values":["3617"]},{"key":["0513","2013"],"values":["9802"]},{"key":["0560","2013"],"values":["5278"]},{"key":["0561","2013"],"values":["11460"]},{"key":["0562","2013"],"values":["20903"]},{"key":["0563","2013"],"values":["7585"]},{"key":["0580","2013"],"values":["150202"]},{"key":["0581","2013"],"values":["133749"]},{"key":["0582","2013"],"values":["14195"]},{"key":["0583","2013"],"values":["42187"]},{"key":["0584","2013"],"values":["7383"]},{"key":["0586","2013"],"values":["26313"]},{"key":["06","2013"],"values":["341235"]},{"key":["0604","2013"],"values":["6375"]},{"key":["0617","2013"],"values":["9406"]},{"key":["0642","2013"],"values":["7039"]},{"key":["0643","2013"],"values":["10975"]},{"key":["0662","2013"],"values":["28713"]},{"key":["0665","2013"],"values":["13209"]},{"key":["0680","2013"],"values":["130798"]},{"key":["0682","2013"],"values":["29516"]},{"key":["0683","2013"],"values":["33155"]},{"key":["0684","2013"],"values":["10969"]},{"key":["0685","2013"],"values":["26419"]},{"key":["0686","2013"],"values":["16464"]},{"key":["0687","2013"],"values":["18197"]},{"key":["07","2013"],"values":["187156"]},{"key":["0760","2013"],"values":["9288"]},{"key":["0761","2013"],"values":["8059"]},{"key":["0763","2013"],"values":["12156"]},{"key":["0764","2013"],"values":["19280"]},{"key":["0765","2013"],"values":["15759"]},{"key":["0767","2013"],"values":["9515"]},{"key":["0780","2013"],"values":["85822"]},{"key":["0781","2013"],"values":["27277"]},{"key":["08","2013"],"values":["233874"]},{"key":["0821","2013"],"values":["5718"]},{"key":["0834","2013"],"values":["6879"]},{"key":["0840","2013"],"values":["14368"]},{"key":["0860","2013"],"values":["13635"]},{"key":["0861","2013"],"values":["12949"]},{"key":["0862","2013"],"values":["8964"]},{"key":["0880","2013"],"values":["63887"]},{"key":["0881","2013"],"values":["19489"]},{"key":["0882","2013"],"values":["26212"]},{"key":["0883","2013"],"values":["35867"]},{"key":["0884","2013"],"values":["15287"]},{"key":["0885","2013"],"values":["10619"]},{"key":["09","2013"],"values":["57161"]},{"key":["0980","2013"],"values":["57161"]},{"key":["10","2013"],"values":["152757"]},{"key":["1060","2013"],"values":["12902"]},{"key":["1080","2013"],"values":["63912"]},{"key":["1081","2013"],"values":["27871"]},{"key":["1082","2013"],"values":["31272"]},{"key":["1083","2013"],"values":["16800"]},{"key":["12","2013"],"values":["1274069"]},{"key":["1214","2013"],"values":["13332"]},{"key":["1230","2013"],"values":["22672"]},{"key":["1231","2013"],"values":["17114"]},{"key":["1233","2013"],"values":["33807"]},{"key":["1256","2013"],"values":["13687"]},{"key":["1257","2013"],"values":["9653"]},{"key":["1260","2013"],"values":["14801"]},{"key":["1261","2013"],"values":["29600"]},{"key":["1262","2013"],"values":["22496"]},{"key":["1263","2013"],"values":["20067"]},{"key":["1264","2013"],"values":["15025"]},{"key":["1265","2013"],"values":["18401"]},{"key":["1266","2013"],"values":["14917"]},{"key":["1267","2013"],"values":["15637"]},{"key":["1270","2013"],"values":["12891"]},{"key":["1272","2013"],"values":["12336"]},{"key":["1273","2013"],"values":["12713"]},{"key":["1275","2013"],"values":["7139"]},{"key":["1276","2013"],"values":["16715"]},{"key":["1277","2013"],"values":["14927"]},{"key":["1278","2013"],"values":["14275"]},{"key":["1280","2013"],"values":["312994"]},{"key":["1281","2013"],"values":["114291"]},{"key":["1282","2013"],"values":["43073"]},{"key":["1283","2013"],"values":["132989"]},{"key":["1284","2013"],"values":["25084"]},{"key":["1285","2013"],"values":["31920"]},{"key":["1286","2013"],"values":["28623"]},{"key":["1287","2013"],"values":["42837"]},{"key":["1290","2013"],"values":["81009"]},{"key":["1291","2013"],"values":["18951"]},{"key":["1292","2013"],"values":["39866"]},{"key":["1293","2013"],"values":["50227"]},{"key":["13","2013"],"values":["306840"]},{"key":["1315","2013"],"values":["10001"]},{"key":["1380","2013"],"values":["94084"]},{"key":["1381","2013"],"values":["23517"]},{"key":["1382","2013"],"values":["41912"]},{"key":["1383","2013"],"values":["59936"]},{"key":["1384","2013"],"values":["77390"]},{"key":["14","2013"],"values":["1615084"]},{"key":["1401","2013"],"values":["35732"]},{"key":["1402","2013"],"values":["36147"]},{"key":["1407","2013"],"values":["12574"]},{"key":["1415","2013"],"values":["24932"]},{"key":["1419","2013"],"values":["15050"]},{"key":["1421","2013"],"values":["15036"]},{"key":["1427","2013"],"values":["8928"]},{"key":["1430","2013"],"values":["10205"]},{"key":["1435","2013"],"values":["12303"]},{"key":["1438","2013"],"values":["4740"]},{"key":["1439","2013"],"values":["6520"]},{"key":["1440","2013"],"values":["28074"]},{"key":["1441","2013"],"values":["39319"]},{"key":["1442","2013"],"values":["11065"]},{"key":["1443","2013"],"values":["8562"]},{"key":["1444","2013"],"values":["5641"]},{"key":["1445","2013"],"values":["5494"]},{"key":["1446","2013"],"values":["6757"]},{"key":["1447","2013"],"values":["5185"]},{"key":["1452","2013"],"values":["11531"]},{"key":["1460","2013"],"values":["9550"]},{"key":["1461","2013"],"values":["8892"]},{"key":["1462","2013"],"values":["12829"]},{"key":["1463","2013"],"values":["33753"]},{"key":["1465","2013"],"values":["10299"]},{"key":["1466","2013"],"values":["9274"]},{"key":["1470","2013"],"values":["15609"]},{"key":["1471","2013"],"values":["13028"]},{"key":["1472","2013"],"values":["10754"]},{"key":["1473","2013"],"values":["8992"]},{"key":["1480","2013"],"values":["533271"]},{"key":["1481","2013"],"values":["61978"]},{"key":["1482","2013"],"values":["42109"]},{"key":["1484","2013"],"values":["14369"]},{"key":["1485","2013"],"values":["53025"]},{"key":["1486","2013"],"values":["12480"]},{"key":["1487","2013"],"values":["37369"]},{"key":["1488","2013"],"values":["56573"]},{"key":["1489","2013"],"values":["38619"]},{"key":["1490","2013"],"values":["105995"]},{"key":["1491","2013"],"values":["23211"]},{"key":["1492","2013"],"values":["12229"]},{"key":["1493","2013"],"values":["23870"]},{"key":["1494","2013"],"values":["38414"]},{"key":["1495","2013"],"values":["18580"]},{"key":["1496","2013"],"values":["52859"]},{"key":["1497","2013"],"values":["8805"]},{"key":["1498","2013"],"values":["12565"]},{"key":["1499","2013"],"values":["31988"]},{"key":["17","2013"],"values":["273815"]},{"key":["1715","2013"],"values":["11810"]},{"key":["1730","2013"],"values":["8426"]},{"key":["1737","2013"],"values":["12013"]},{"key":["1760","2013"],"values":["4131"]},{"key":["1761","2013"],"values":["15136"]},{"key":["1762","2013"],"values":["3656"]},{"key":["1763","2013"],"values":["11292"]},{"key":["1764","2013"],"values":["8925"]},{"key":["1765","2013"],"values":["9953"]},{"key":["1766","2013"],"values":["13011"]},{"key":["1780","2013"],"values":["87786"]},{"key":["1781","2013"],"values":["23949"]},{"key":["1782","2013"],"values":["10563"]},{"key":["1783","2013"],"values":["12071"]},{"key":["1784","2013"],"values":["25817"]},{"key":["1785","2013"],"values":["15276"]},{"key":["18","2013"],"values":["285395"]},{"key":["1814","2013"],"values":["7289"]},{"key":["1860","2013"],"values":["5580"]},{"key":["1861","2013"],"values":["15267"]},{"key":["1862","2013"],"values":["9500"]},{"key":["1863","2013"],"values":["6982"]},{"key":["1864","2013"],"values":["4875"]},{"key":["1880","2013"],"values":["140599"]},{"key":["1881","2013"],"values":["20904"]},{"key":["1882","2013"],"values":["11096"]},{"key":["1883","2013"],"values":["29728"]},{"key":["1884","2013"],"values":["10399"]},{"key":["1885","2013"],"values":["23176"]},{"key":["19","2013"],"values":["259054"]},{"key":["1904","2013"],"values":["4411"]},{"key":["1907","2013"],"values":["9834"]},{"key":["1960","2013"],"values":["8175"]},{"key":["1961","2013"],"values":["15524"]},{"key":["1962","2013"],"values":["5608"]},{"key":["1980","2013"],"values":["142131"]},{"key":["1981","2013"],"values":["21769"]},{"key":["1982","2013"],"values":["12872"]},{"key":["1983","2013"],"values":["25237"]},{"key":["1984","2013"],"values":["13493"]},{"key":["20","2013"],"values":["277349"]},{"key":["2021","2013"],"values":["6730"]},{"key":["2023","2013"],"values":["10061"]},{"key":["2026","2013"],"values":["10023"]},{"key":["2029","2013"],"values":["15157"]},{"key":["2031","2013"],"values":["10766"]},{"key":["2034","2013"],"values":["6849"]},{"key":["2039","2013"],"values":["7096"]},{"key":["2061","2013"],"values":["10691"]},{"key":["2062","2013"],"values":["19998"]},{"key":["2080","2013"],"values":["56767"]},{"key":["2081","2013"],"values":["50023"]},{"key":["2082","2013"],"values":["10873"]},{"key":["2083","2013"],"values":["15021"]},{"key":["2084","2013"],"values":["21582"]},{"key":["2085","2013"],"values":["25712"]},{"key":["21","2013"],"values":["277970"]},{"key":["2101","2013"],"values":["5785"]},{"key":["2104","2013"],"values":["9511"]},{"key":["2121","2013"],"values":["11354"]},{"key":["2132","2013"],"values":["9491"]},{"key":["2161","2013"],"values":["18931"]},{"key":["2180","2013"],"values":["97236"]},{"key":["2181","2013"],"values":["37250"]},{"key":["2182","2013"],"values":["25442"]},{"key":["2183","2013"],"values":["26141"]},{"key":["2184","2013"],"values":["36829"]},{"key":["22","2013"],"values":["242156"]},{"key":["2260","2013"],"values":["9548"]},{"key":["2262","2013"],"values":["18062"]},{"key":["2280","2013"],"values":["24509"]},{"key":["2281","2013"],"values":["96978"]},{"key":["2282","2013"],"values":["18450"]},{"key":["2283","2013"],"values":["19623"]},{"key":["2284","2013"],"values":["54986"]},{"key":["23","2013"],"values":["126461"]},{"key":["2303","2013"],"values":["5458"]},{"key":["2305","2013"],"values":["6559"]},{"key":["2309","2013"],"values":["14643"]},{"key":["2313","2013"],"values":["11984"]},{"key":["2321","2013"],"values":["10420"]},{"key":["2326","2013"],"values":["7160"]},{"key":["2361","2013"],"values":["10281"]},{"key":["2380","2013"],"values":["59956"]},{"key":["24","2013"],"values":["261112"]},{"key":["2401","2013"],"values":["7006"]},{"key":["2403","2013"],"values":["2436"]},{"key":["2404","2013"],"values":["5344"]},{"key":["2409","2013"],"values":["6738"]},{"key":["2417","2013"],"values":["4175"]},{"key":["2418","2013"],"values":["3155"]},{"key":["2421","2013"],"values":["5954"]},{"key":["2422","2013"],"values":["2595"]},{"key":["2425","2013"],"values":["2757"]},{"key":["2460","2013"],"values":["8583"]},{"key":["2462","2013"],"values":["6887"]},{"key":["2463","2013"],"values":["2875"]},{"key":["2480","2013"],"values":["118349"]},{"key":["2481","2013"],"values":["12270"]},{"key":["2482","2013"],"values":["71988"]},{"key":["25","2013"],"values":["249436"]},{"key":["2505","2013"],"values":["6471"]},{"key":["2506","2013"],"values":["2980"]},{"key":["2510","2013"],"values":["5066"]},{"key":["2513","2013"],"values":["3436"]},{"key":["2514","2013"],"values":["16387"]},{"key":["2518","2013"],"values":["4709"]},{"key":["2521","2013"],"values":["6299"]},{"key":["2523","2013"],"values":["18339"]},{"key":["2560","2013"],"values":["8168"]},{"key":["2580","2013"],"values":["75383"]},{"key":["2581","2013"],"values":["41278"]},{"key":["2582","2013"],"values":["27838"]},{"key":["2583","2013"],"values":["9886"]},{"key":["2584","2013"],"values":["23196"]}]} 

EDIT2: Работает при копировании-pasteing

Итак, если я просто распечатаю r.text в терминале, скопируйте текст и запустите json.loads(text в интерпретаторе Python, он работает как ожидалось. Может быть ошибка с файлом?

EDIT3: Simplejson ошибка

Если я использую simplejson модуль вместо этого, я получаю более описательное сообщение об ошибке:

simplejson.scanner.JSONDecodeError: Expecting value: line 1 column 1 (char 0) 

Answer 1

Это неисправного из-за в выходе на неэкранированных \r. Я скопировал свой полный вывод в строку p, и попытался запустить его через json.loads:

>>> json.loads(p) 
Traceback (most recent call last): 
    File "<stdin>", line 1, in <module> 
    File "/usr/lib/python2.7/json/__init__.py", line 338, in loads 
    return _default_decoder.decode(s) 
    File "/usr/lib/python2.7/json/decoder.py", line 366, in decode 
    obj, end = self.raw_decode(s, idx=_w(s, 0).end()) 
    File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode 
    obj, end = self.scan_once(s, idx) 
ValueError: Invalid control character at: line 1 column 350 (char 349) 
>>> p[349] 
'\r' 

Вот еще контекст вокруг \r:

>>> p[250:355] 
' Heby kommun. Observera att l\xc3\xa4nssiffrorna inte \xc3\xa4r j\xc3\xa4mf\xc3\xb6rbara med l\xc3\xa4nssiffrorna bak\xc3\xa5t i tiden.\r\n"},{' 

Вы можете это исправить, спасаясь от \r (а также \n, который также показывает вверх):

>>> x = p.replace("\r", "\\r").replace('\n', '\\n') 
>>> json.loads(x) 
<the huge json object prints here> 

Answer 2

Я просто наткнулся на это, это annoyin g - отлично работает в IPython, отлично работает, если вы сохраняете текст ответа в файл и загружаете его, и единственные проблемы, связанные с этим, которые я мог найти, - это те, которые имеют неправильные заголовки Content-Type, которых у вас нет.

То, что я обнаружил, что работал использовал

return json.loads(str(req.content, "utf-8")) 

, кажется, работает, и это менее уродливые, чем писать req.text в файл, а затем загрузку JSON оттуда;)