o TjD@stddlmZddlmZmZddlZddlmZddl m Z ddl m Z ddl mZddlmZddlZdd lmZmZdd lmZdd lmZmZdd lmZmZmZmZdd lm Z m!Z!m"Z"erhddlm#Z#ej$ej%ej&ej'ej(ej)ej)dZ*ej&ej+dfej)ej,e fej$ej-dfej%ej-dfej'ej-dfej.ej,dfej(ej/dfiZ0ej-dej+dej,diZ1GdddeZ2dS)) annotations) TYPE_CHECKINGAnyN) infer_dtype)iNaT)NoBufferPresent)cache_readonly)BaseMaskedDtype) ArrowDtypeDatetimeTZDtype)is_string_dtype) PandasBufferPandasBufferPyarrow)Column ColumnBuffersColumnNullType DtypeKind) ArrowCTypes Endiannessdtype_to_arrow_c_fmt)Buffer)iufbUMmzThis column is non-nullablezThis column uses NaN as nullz!This column uses a sentinel valuec@seZdZdZd1d2d d Zd3d d Zed3ddZed4ddZ d4ddZ eddZ eddZ ed3ddZ ed5ddZd3ddZd6d7d#d$Zd8d&d'Zd9d)d*Zd:d,d-Zd;d/d0Zd S)< PandasColumna A column object, with only the methods and properties required by the interchange protocol defined. A column can contain one or more chunks. Each chunk can contain up to three buffers - a data buffer, a mask buffer (depending on null representation), and an offsets buffer (if variable-size binary; e.g., variable-length strings). Note: this Column object can only be produced by ``__dataframe__``, so doesn't need its own version or ``__column__`` protocol. Tcolumn pd.Series allow_copyboolreturnNonecCsNt|tjrtd|jdt|tjstdt|d||_||_ dS)zu Note: doesn't deal with extension arrays yet, just assume a regular Series/ndarray for now. zExpected a Series, got a DataFrame. This likely happened because you called __dataframe__ on a DataFrame which, after converting column names to string, resulted in duplicated names: zD. Please rename these columns before using the interchange protocol.zColumns of type  not handled yetN) isinstancepd DataFrame TypeErrorcolumnsSeriesNotImplementedErrortype_col _allow_copy)selfr r"r2a/home/dinkstrade/pdmp-scanner/venv/lib/python3.10/site-packages/pandas/core/interchange/column.py__init__Ts   zPandasColumn.__init__intcCs|jjS)z2 Size of the column, in elements. )r/sizer1r2r2r3r6hszPandasColumn.sizecCdS)z7 Offset of first element. Always zero. rr2r7r2r2r3offsetnszPandasColumn.offsettuple[DtypeKind, int, str, str]cCs~|jj}t|tjr!|jjj}||j\}}}}tj ||t j fSt |r:t |jdvr6tjdt|t j fStd||S)N)stringemptyz.Non-string object dtypes are not supported yet)r/dtyper'r(CategoricalDtypevaluescodes_dtype_from_pandasdtyper CATEGORICALrNATIVEr rSTRINGrr-)r1r>rA_bitwidthc_arrow_dtype_f_strr2r2r3r>vs.    zPandasColumn.dtypecCst|jd}|durtd|dt|tr|jj}nt|tr'|j j}n t|t r1|jj}n|j}|dkr@||j t j |fS||j dt||fS)z/ See `self.dtype` for details. N Data type z& not supported by interchange protocolz bool[pyarrow]r=) _NP_KINDSgetkind ValueErrorr'r numpy_dtype byteorderr baser itemsizerBOOLr)r1r>rLrOr2r2r3rBs"      z$PandasColumn._dtype_from_pandasdtypecCs:|jdtjks td|jjjdtt |jjj dS)a: If the dtype is categorical, there are two options: - There are only values in the data buffer. - There is a separate non-categorical Column encoding for categorical values. Raises TypeError if the dtype is not categorical Content of returned dict: - "is_ordered" : bool, whether the ordering of dictionary indices is semantically meaningful. - "is_dictionary" : bool, whether a dictionary-style mapping of categorical values to other objects exists - "categories" : Column representing the (implicit) mapping of indices to category values (e.g. an array of cat1, cat2, ...). None if not a dictionary-style categorical. rzCdescribe_categorical only works on a column with categorical dtype!T) is_ordered is_dictionary categories) r>rrCr*r/catorderedrr(r,rUr7r2r2r3describe_categoricalsz!PandasColumn.describe_categoricalcCst|jjtrtj}d}||fSt|jjtr/|jjjj d ddur*tj dfStj dfS|jd}z t |\}}W||fStyNtd|dw)NrrIz not yet supported)r'r/r>r r USE_BYTEMASKr array _pa_arraychunksbuffers NON_NULLABLE USE_BITMASK_NULL_DESCRIPTIONKeyErrorr-)r1column_null_dtype null_valuerLnullvaluer2r2r3 describe_nulls    zPandasColumn.describe_nullcCs|jS)zB Number of null elements. Should always be known. )r/isnasumitemr7r2r2r3 null_countszPandasColumn.null_countdict[str, pd.Index]cCs d|jjiS)z8 Store specific metadata of the column. z pandas.index)r/indexr7r2r2r3metadatas zPandasColumn.metadatacCr8)zE Return the number of chunks the column consists of. rYr2r7r2r2r3 num_chunksszPandasColumn.num_chunksNn_chunks int | Noneccsv|r6|dkr6t|j}||}||dkr|d7}td|||D]}t|jj||||jVq"dS|VdS)zy Return an iterator yielding the chunks. See `DataFrame.get_chunks` for details on ``n_chunks``. rYrN)lenr/rangerilocr0)r1rpr6stepstartr2r2r3 get_chunkss    zPandasColumn.get_chunksrcCs\|ddd}z||d<Wn tyYnwz ||d<W|Sty-Y|Sw)a` Return a dictionary containing the underlying buffers. The returned dictionary has the following contents: - "data": a two-element tuple whose first element is a buffer containing the data and whose second element is the data buffer's associated dtype. - "validity": a two-element tuple whose first element is a buffer containing mask values indicating missing data and whose second element is the mask value buffer's associated dtype. None if the null representation is not a bit or byte mask. - "offsets": a two-element tuple whose first element is a buffer containing the offset values for variable-size binary data (e.g., variable-length strings) and whose second element is the offsets buffer's associated dtype. None if the data buffer does not have an associated offsets buffer. N)datavalidityoffsetsryrz)_get_data_buffer_get_validity_bufferr_get_offsets_buffer)r1r^r2r2r3 get_buffers s   zPandasColumn.get_buffers.tuple[Buffer, tuple[DtypeKind, int, str, str]]c Cs|jdtjtjtjtjtjfvri|j}|jdtjkr/t|jddkr/|jj d }n/|jj }t |jjtr>|j}n t |jjtr[|jjd}t|dt|d}||fS|j}t||jd}||fS|jdtjkr|jjj}t||jd}||j}||fS|jdtjkr|j }t}|D]}t |tr||j dd qtt!j"|d d }|j}||fSt#d |jjd )zZ Return the buffer containing the data and the buffer's associated dtype. rNrYlength)r"utf-8encodinguint8)r>rIr&)$r>rINTUINTFLOATrRDATETIMErrr/dt tz_convertto_numpyr[r'r _datar r\r]rr^_ndarrayr r0rCr@_codesrBrE bytearraystrextendencodenp frombufferr-) r1r>np_arrarrbufferrAbufrobjr2r2r3r{0sN "      zPandasColumn._get_data_buffertuple[Buffer, Any] | Nonec Cs`|j\}}t|jjtr7|jjjjd}tj dt j t j f}| ddur'dSt| dt|d}||fSt|jjtrT|jjj}t|}tj dt j t j f}||fS|jdtjkr|j}|dk}| }tjt|ftjd}t|D]\} } t| tr|n||| <qwt|}tj dt j t j f}||fSz t|d} Wt| tytdw) z Return the buffer containing the mask values indicating missing data and the buffer's associated dtype. Raises NoBufferPresent if null representation is not a bit or byte mask. rrYNrr=shaper>z! so does not have a separate maskzSee self.describe_null)rgr'r/r>r r[r\r]rrRrrrDr^rrrr _maskr rErrzerosbool_ enumerater_NO_VALIDITY_BUFFERrbr-r) r1reinvalidrr>rmaskrvalidrrmsgr2r2r3r|ns@     z!PandasColumn._get_validity_buffertuple[PandasBuffer, Any]c Cs|jdtjkrM|j}d}tjt|dftjd}t |D]\}}t |t r5|j dd}|t|7}|||d<q t |}tjdtjtjf}||fStd)a Return the buffer containing the offset values for variable-size binary data (e.g., variable-length strings) and the buffer's associated dtype. Raises NoBufferPresent if the data buffer does not have an associated offsets buffer. rrYrrr@zJThis column has a fixed-length dtype so it does not have an offsets buffer)r>rrEr/rrrrrint64rr'rrr rrINT64rrDr) r1r@ptrrzrvrrr>r2r2r3r}s&     z PandasColumn._get_offsets_buffer)T)r r!r"r#r$r%)r$r5)r$r:)r$rl)N)rprq)r$r)r$r)r$r)r$r)__name__ __module__ __qualname____doc__r4r6propertyr9rr>rBrXrgrkrnrorwr~r{r|r}r2r2r2r3rHs.    !        % >9r)3 __future__rtypingrrnumpyrpandas._libs.librpandas._libs.tslibsr pandas.errorsrpandas.util._decoratorsrpandas.core.dtypes.dtypesr pandasr(r r pandas.api.typesr pandas.core.interchange.bufferr r*pandas.core.interchange.dataframe_protocolrrrrpandas.core.interchange.utilsrrrrrrrrRrErrJUSE_NAN USE_SENTINELr_rCrZrarrr2r2r2r3sJ