我正在尝试使用 Python Pandas 导入 CSV 文件。此文件中的示例数据如下，其中第一行是用逗号分隔的列名。
End Customer Organization ID,End Customer Organization Name,End Customer Top Parent Organization ID,End Customer Top Parent Organization Name,Reseller Top Parent ID,Reseller Top Parent Name,Business,Rev Sum Division,Rev Sum Category,Product Family,Version,Pricing Level,Summary Pricing Level,Detail Pricing Level,MS Sales Amount,MS Sales Licenses,Fiscal Year,Sales Date
11027676,Baroda Western Uttar Pradesh Gramin Bankgfhgfnjgfnmjmhgmghmghmghmnghnmghnmhgnmghnghngh,4078446,Bank Of Barodadfhhgfjyjtkyukujkyujkuhykluiluilui;iooi';po'fserwefvegwegf,1809012,"Hcl Infosystems Ltd - Partnerdghftrutyhb frhywer5y5tyu6ui7iukluyj,lgjmfgnhfrgweffw",Server & CALsdgrgrfgtrhytrnhjdgthjtyjkukmhjmghmbhmgfngdfbndfhtgh,SQL Server & CALdfhtrhtrgbhrghrye5y45y45yu56juhydsgfaefwe,SQL CALdhdfthtrutrjurhjethfdehrerfgwerweqeadfawrqwerwegtrhyjuytjhyj,SQL CALdtrye45y3t434tjkabcjkasdhfhasdjkcbaksmjcbfuigkjasbcjkasbkdfhiwh,2005,Openfkvgjesropiguwe90fujklascnioawfy98eyfuiasdbcvjkxsbhg,Open Lklbjdfoigueroigbjvwioergyuiowerhgosdhvgfoisdhyguiserhguisrh,"Open Stddfm,vdnoghioerivnsdflierohgushdfovhsiodghuiohdbvgsjdhgouiwerho",125.85,1,FY07,12/28/2006
12835756,Uttam Strips Pvt Ltd,12835756,Uttam Strips Pvt Ltd,12565538,Redington C/O Fortis Financial Services Ltd,MBS,Dynamics ERP,Dynamics NAV,Dynamics NAV Business Essentials,Non-specific,Other,MBS SA,MBS New Customer Enhanc. Def,0,0,FY09,9/15/2008
12233135,Bhagwan Singh Tondon,12233135,Bhagwan Singh Tondon,2652941,H B S Systems Pvt Ltd,Server & CAL,SQL Server & CAL,SQL CAL,SQL CAL,Non-specific,Open,Open L&SA,Deferred Open L&SA - New,0,0,FY09,9/15/2008
11602305,Maya Academy Of Advanced Cinematics,9750934,Maya Entertainment Ltd,336146,Embee Software Pvt Ltd,Server & CAL,Windows Server & CAL,Windows Server HPC,Windows Compute Cluster Server,Non-specific,Open,Open V/MYO - Rec,OLV Perpet L&SA Recur-Def,0,0,FY09,9/25/2008
13336009,Remiel Softech Solution Pvt Ltd,13336009,Remiel Softech Solution Pvt Ltd,13335482,Redington C/O Remiel Softech Solutions Pvt Ltd,MBS,Dynamics ERP,Dynamics NAV,Dynamics NAV Business Essentials,Non-specific,Other,MBS SA,MBS New Customer Enhanc. Def,0,0,FY09,12/23/2008
我正在使用以下代码导入:
import pandas as pd
df=pd.read_csv('file path.csv',sep=',')
它给出了以下错误:
Traceback (most recent call last):
File "<pyshell#25>", line 1, in <module>
df=pd.read_csv(filename,sep=',')
File "C:\Python33\lib\site-packages\pandas\io\parsers.py", line 400, in parser_f
return _read(filepath_or_buffer, kwds)
File "C:\Python33\lib\site-packages\pandas\io\parsers.py", line 205, in _read
return parser.read()
File "C:\Python33\lib\site-packages\pandas\io\parsers.py", line 608, in read
ret = self._engine.read(nrows)
File "C:\Python33\lib\site-packages\pandas\io\parsers.py", line 1028, in read
data = self._reader.read(nrows)
File "parser.pyx", line 706, in pandas.parser.TextReader.read (pandas\parser.c:6745)
File "parser.pyx", line 728, in pandas.parser.TextReader._read_low_memory (pandas\parser.c:6964)
File "parser.pyx", line 804, in pandas.parser.TextReader._read_rows (pandas\parser.c:7780)
File "parser.pyx", line 890, in pandas.parser.TextReader._convert_column_data (pandas\parser.c:8793)
File "parser.pyx", line 950, in pandas.parser.TextReader._convert_tokens (pandas\parser.c:9484)
File "parser.pyx", line 1026, in pandas.parser.TextReader._convert_with_dtype (pandas\parser.c:10642)
File "parser.pyx", line 1046, in pandas.parser.TextReader._string_convert (pandas\parser.c:10853)
File "parser.pyx", line 1278, in pandas.parser._string_box_utf8 (pandas\parser.c:15657)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc1 in position 90: invalid start byte
因为它看起来像是一个 Unicode 错误，所以我这次运行时更改了编码:
df=pd.read_csv(filename,encoding='utf-16',sep=',')
它给出了以下错误:
Traceback (most recent call last):
File "<pyshell#26>", line 1, in <module>
df=pd.read_csv(filename,encoding='utf-16',sep=',')
File "C:\Python33\lib\site-packages\pandas\io\parsers.py", line 400, in parser_f
return _read(filepath_or_buffer, kwds)
File "C:\Python33\lib\site-packages\pandas\io\parsers.py", line 198, in _read
parser = TextFileReader(filepath_or_buffer, **kwds)
File "C:\Python33\lib\site-packages\pandas\io\parsers.py", line 479, in __init__
self._make_engine(self.engine)
File "C:\Python33\lib\site-packages\pandas\io\parsers.py", line 586, in _make_engine
self._engine = CParserWrapper(self.f, **self.options)
File "C:\Python33\lib\site-packages\pandas\io\parsers.py", line 957, in __init__
self._reader = _parser.TextReader(src, **kwds)
File "parser.pyx", line 477, in pandas.parser.TextReader.__cinit__ (pandas\parser.c:4434)
File "parser.pyx", line 592, in pandas.parser.TextReader._get_header (pandas\parser.c:5660)
File "parser.pyx", line 768, in pandas.parser.TextReader._tokenize_rows (pandas\parser.c:7451)
File "parser.pyx", line 1661, in pandas.parser.raise_parser_error (pandas\parser.c:18744)
pandas.parser.CParserError: Error tokenizing data. C error: Calling read(nbytes) on source failed. Try engine='python'.
不确定为什么会这样？甚至尝试使用 Text to Columns 将 CSV 文件转换为 Excel，并使用 Pandas 的 read_excel 函数。这也给出了错误(如下):
Traceback (most recent call last):
File "<pyshell#30>", line 1, in <module>
df=pd.read_excel('J:\dmqp on 192.168.1.41\MS Sales Dump (FY09)xls','MS Sales Dump (FY09)')
File "C:\Python33\lib\site-packages\pandas\io\excel.py", line 52, in read_excel
return ExcelFile(path_or_buf,kind=kind).parse(sheetname=sheetname,
File "C:\Python33\lib\site-packages\pandas\io\excel.py", line 68, in __init__
import xlrd # throw an ImportError if we need to
ImportError: No module named 'xlrd'
有人可以帮助解决上述错误以及在导入 CSV 和 Excel 时这里出了什么问题。
我试过这段编码改变的代码:
df=pd.read_csv(filename,encoding='iso-8859-1',sep=',')
它没有给出任何错误，但作为一列导入而不是将其分解为单独的列。
>>>df
<class 'pandas.core.frame.DataFrame'>
Int64Index: 263244 entries, 0 to 263243
Data columns (total 1 columns):
End Customer Organization ID,End Customer Organization Name,End Customer Top Parent Organization ID,End Customer Top Parent Organization Name,Reseller Top Parent ID,Reseller Top Parent Name,Business,Rev Sum Division,Rev Sum Category,Product Family,Version,Pricing Level,Summary Pricing Level,Detail Pricing Level,MS Sales Amount,MS Sales Licenses,Fiscal Year,Sales Date 263244 non-null values
dtypes: object(1)
通过将上面的示例数据存储在文本文件中并导入来检查上面的示例数据，这是我得到的输出:
>>> df =pd.read_csv(r'J:\Data.txt')
>>> print(df)
End Customer Organization ID \
0 11027676
1 12835756
2 12233135
3 11602305
4 13336009
End Customer Organization Name \
0 Baroda Western Uttar Pradesh Gramin Bankgfhgfn...
1 Uttam Strips Pvt Ltd
2 Bhagwan Singh Tondon
3 Maya Academy Of Advanced Cinematics
4 Remiel Softech Solution Pvt Ltd
End Customer Top Parent Organization ID \
0 4078446
1 12835756
2 12233135
3 9750934
4 13336009
End Customer Top Parent Organization Name Reseller Top Parent ID \
0 Bank Of Barodadfhhgfjyjtkyukujkyujkuhykluiluil... 1809012
1 Uttam Strips Pvt Ltd 12565538
2 Bhagwan Singh Tondon 2652941
3 Maya Entertainment Ltd 336146
4 Remiel Softech Solution Pvt Ltd 13335482
Reseller Top Parent Name \
0 Hcl Infosystems Ltd - Partnerdghftrutyhb frhyw...
1 Redington C/O Fortis Financial Services Ltd
2 H B S Systems Pvt Ltd
3 Embee Software Pvt Ltd
4 Redington C/O Remiel Softech Solutions Pvt Ltd
Business \
0 Server & CALsdgrgrfgtrhytrnhjdgthjtyjkukmhjmgh...
1 MBS
2 Server & CAL
3 Server & CAL
4 MBS
Rev Sum Division \
0 SQL Server & CALdfhtrhtrgbhrghrye5y45y45yu56ju...
1 Dynamics ERP
2 SQL Server & CAL
3 Windows Server & CAL
4 Dynamics ERP
Rev Sum Category \
0 SQL CALdhdfthtrutrjurhjethfdehrerfgwerweqeadfa...
1 Dynamics NAV
2 SQL CAL
3 Windows Server HPC
4 Dynamics NAV
Product Family Version \
0 SQL CALdtrye45y3t434tjkabcjkasdhfhasdjkcbaksmj... 2005
1 Dynamics NAV Business Essentials Non-specific
2 SQL CAL Non-specific
3 Windows Compute Cluster Server Non-specific
4 Dynamics NAV Business Essentials Non-specific
Pricing Level \
0 Openfkvgjesropiguwe90fujklascnioawfy98eyfuiasd...
1 Other
2 Open
3 Open
4 Other
Summary Pricing Level \
0 Open Lklbjdfoigueroigbjvwioergyuiowerhgosdhvgf...
1 MBS SA
2 Open L&SA
3 Open V/MYO - Rec
4 MBS SA
Detail Pricing Level MS Sales Amount \
0 Open Stddfm,vdnoghioerivnsdflierohgushdfovhsio... 125.85
1 MBS New Customer Enhanc. Def 0.00
2 Deferred Open L&SA - New 0.00
3 OLV Perpet L&SA Recur-Def 0.00
4 MBS New Customer Enhanc. Def 0.00
MS Sales Licenses Fiscal Year Sales Date
0 1 FY07 12/28/2006
1 0 FY09 9/15/2008
2 0 FY09 9/15/2008
3 0 FY09 9/25/2008
4 0 FY09 12/23/2008
>>>
这是在每一列之后添加'\'，并且列名不是一个接着一个。相反，它们似乎在导入每一列后都在新行上。
最佳答案
我想您的主要问题与编码有关。我曾遭受过处理 csv 文件中奇怪编码的痛苦。在这些情况下对我有帮助的是尝试检测文件的真实编码并使用 pandas 正确加载它。
试试下一段代码:
from chardet.universaldetector import UniversalDetector
def test_encoding(file_name):
detector = UniversalDetector()
with open(file_name, 'rb') as f:
for line in f:
detector.feed(line)
if detector.done:
break
detector.close()
r = detector.result
return "Detected encoding %s with confidence %s" % (r['encoding'], r['confidence'])
# pass the file path in the function to see result
test_encoding('C:\Users\..\file.csv')
输出:
'Detected encoding UTF-16 with confidence 1.0'
这将尝试推断文件的编码，然后您可以尝试使用 pandas 正确加载它。 希望对您有所帮助...
关于python - Pandas导入CSV和Excel文件报错，我们在Stack Overflow上找到一个类似的问题： https://stackoverflow.com/questions/19293316/
我有一个Ruby程序，它使用rubyzip压缩XML文件的目录树。gem。我的问题是文件开始变得很重，我想提高压缩级别，因为压缩时间不是问题。我在rubyzipdocumentation中找不到一种为创建的ZIP文件指定压缩级别的方法。有人知道如何更改此设置吗？是否有另一个允许指定压缩级别的Ruby库？ 最佳答案 这是我通过查看rubyzip内部创建的代码。level=Zlib::BEST_COMPRESSIONZip::ZipOutputStream.open(zip_file)do|zip|Dir.glob("**/*")d
我试图在一个项目中使用rake，如果我把所有东西都放到Rakefile中，它会很大并且很难读取/找到东西，所以我试着将每个命名空间放在lib/rake中它自己的文件中，我添加了这个到我的rake文件的顶部:Dir['#{File.dirname(__FILE__)}/lib/rake/*.rake'].map{|f|requiref}它加载文件没问题，但没有任务。我现在只有一个.rake文件作为测试，名为“servers.rake”，它看起来像这样:namespace:serverdotask:testdoputs"test"endend所以当我运行rakeserver:testid时
我的目标是转换表单输入，例如“100兆字节”或“1GB”，并将其转换为我可以存储在数据库中的文件大小(以千字节为单位)。目前，我有这个:defquota_convert@regex=/([0-9]+)(.*)s/@sizes=%w{kilobytemegabytegigabyte}m=self.quota.match(@regex)if@sizes.include?m[2]eval("self.quota=#{m[1]}.#{m[2]}")endend这有效，但前提是输入是倍数(“gigabytes”，而不是“gigabyte”)并且由于使用了eval看起来疯狂不安全。所以，功能正常，
关闭。这个问题是opinion-based.它目前不接受答案。想要改进这个问题？更新问题，以便editingthispost可以用事实和引用来回答它.关闭4年前。Improvethisquestion我想在固定时间创建一系列低音和高音调的哔哔声。例如:在150毫秒时发出高音调的蜂鸣声在151毫秒时发出低音调的蜂鸣声200毫秒时发出低音调的蜂鸣声250毫秒的高音调蜂鸣声有没有办法在Ruby或Python中做到这一点？我真的不在乎输出编码是什么(.wav、.mp3、.ogg等等)，但我确实想创建一个输出文件。
Rails2.3可以选择随时使用RouteSet#add_configuration_file添加更多路由。是否可以在Rails3项目中做同样的事情？ 最佳答案 在config/application.rb中:config.paths.config.routes在Rails3.2(也可能是Rails3.1)中，使用:config.paths["config/routes"] 关于ruby-on-rails-Rails3中的多个路由文件，我们在StackOverflow上找到一个类似的问题
对于具有离线功能的智能手机应用程序，我正在为Xml文件创建单向文本同步。我希望我的服务器将增量/差异(例如GNU差异补丁)发送到目标设备。这是计划:Time=0Server:hasversion_1ofXmlfile(~800kiB)Client:hasversion_1ofXmlfile(~800kiB)Time=1Server:hasversion_1andversion_2ofXmlfile(each~800kiB)computesdeltaoftheseversions(=patch)(~10kiB)sendspatchtoClient(~10kiBtransferred)Cl
我正在寻找执行以下操作的正确语法(在Perl、Shell或Ruby中):#variabletoaccessthedatalinesappendedasafileEND_OF_SCRIPT_MARKERrawdatastartshereanditcontinues. 最佳答案 Perl用__DATA__做这个:#!/usr/bin/perlusestrict;usewarnings;while(){print;}__DATA__Texttoprintgoeshere 关于ruby-如何将脚
使用带有Rails插件的vim，您可以创建一个迁移文件，然后一次性打开该文件吗？textmate也可以这样吗？ 最佳答案 你可以使用rails.vim然后做类似的事情::Rgeneratemigratonadd_foo_to_bar插件将打开迁移生成的文件，这正是您想要的。我不能代表textmate。 关于ruby-使用VimRails，您可以创建一个新的迁移文件并一次性打开它吗？，我们在StackOverflow上找到一个类似的问题： https://sta
我正在使用ruby1.9解析以下带有MacRoman字符的csv文件#encoding:ISO-8859-1#csv_parse.csvName,main-dialogue"Marceu","Giveittohimóhe,hiswife."我做了以下解析。require'csv'input_string=File.read("../csv_parse.rb").force_encoding("ISO-8859-1").encode("UTF-8")#=>"Name,main-dialogue\r\n\"Marceu\",\"Giveittohim\x97he,hiswife.\"\
查看Ruby的CSV库的文档，我非常确定这是可能且简单的。我只需要使用Ruby删除CSV文件的前三列，但我没有成功运行它。 最佳答案 csv_table=CSV.read(file_path_in,:headers=>true)csv_table.delete("header_name")csv_table.to_csv#=>ThenewCSVinstringformat检查CSV::Table文档:http://ruby-doc.org/stdlib-1.9.2/libdoc/csv/rdoc/CSV/Table.html