TypeError when trying to add known marker genes

[oligomyeggo]

Hello, I am working through your latest iteration of single cell tutorial and having issues plotting gene expression of known marker gene expression profiles. I made a dictionary of marker_genes, and used the following code for each of my marker gene groups:

ids_celltypeA = np.in1d(adata.var_names, marker_genes['celltypeA'])
ids_celltypeB = np.in1d(adata.var_names, marker_genes['celltypeB'])

adata.obs['celltypeA_marker_expr'] = adata.X[:,ids_celltypeA].mean(1)
adata.obs['celltypeB_marker_expr'] = adata.X[:,ids_celltypeB].mean(1)

This worked without issue, but when I attempted to plot sc.pl.umap(adata, color=['celltypeA_marker_expr', 'celltypeB_marker_expr']), I received the following error:

IndexError: index 288 is out of bounds for axis 0 with size 1

I looked at each step of the code above, and the sizes of everything are correct with respect to my data (right number of genes, right number of cells). When I tried to inspect adata.obs['celltypeA_marker_expr'], I got this error:

TypeError: cannot concatenate object of type '<class 'numpy.ndarray'>'; only Series and DataFrame objs are valid

I am not sure what's going on, especially since I have been able to use this process on several analysis workflows before. The only thing that I can think of is that maybe something changed with the scanpy version? I am currently using scanpy v1.8.1 (and pandas v1.3.2). My last successful iteration of using the above code was scanpy v1.7.1 (and pandas v1.2.4).

[oligomyeggo]

Messing around with things, I seem to have found a workaround. Instead of calling:

adata.obs['celltypeA_marker_expr'] = adata.X[:,ids_celltypeA].mean(1)

If I call:

adata.obs['celltypeA_marker_expr'] = np.ravel(adata.X[:,ids_celltypeA].mean(1))

Things now work as expected. I am still not really sure what is going on, but I am guessing it's something to do with updated package versions. If anyone has any insight or a better solution than adding np.ravel I would love to know!

[LuckyMD]

Hi @oligomyeggo ,

I'm afraid I can't give you more insight there on pandas updates. But maybe you would get a better answer on the scanpy github?

[oligomyeggo]

Hi @LuckyMD, no problem! I'll ask over on the scanpy github. Thank you!

[catsargent]

Hi, I am also running into this issue. Did you find a solution?

[LuckyMD]

If you report the whole error traceback, it would be easier to assess.

[catsargent]

It appears to be the same as this issue: scverse/scanpy#2008

[LuckyMD]

Great, then it's being looked into. Just to leave this here for anyone else having this issue... a temporary solution is to downgrade to pandas <1.3