Skip to content

Tests leave load of junk in my temp-dir #298

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
lurch opened this issue Jun 11, 2019 · 12 comments
Closed

Tests leave load of junk in my temp-dir #298

lurch opened this issue Jun 11, 2019 · 12 comments
Assignees

Comments

@lurch
Copy link
Contributor

lurch commented Jun 11, 2019

Observe:

$ ls -l /tmp/tmp*
ls: cannot access /tmp/tmp*: No such file or directory
$ nosetests tests/test_tempfs.py
.......................................................................S....................................................................................S...........
----------------------------------------------------------------------
Ran 168 tests in 1.108s

OK (SKIP=2)
$ ls -l /tmp/tmp*
/tmp/tmp0_2hqgqlfstestosfs:
total 0

/tmp/tmp_0fkvyxqfstestosfs:
total 0

/tmp/tmp15_h_uavfsTempFS:
total 12
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 egg
drwxrwxr-x 3 andrew andrew 4096 Jun 11 02:03 foo
-rw-rw-r-- 1 andrew andrew   12 Jun 11 02:03 top.txt

/tmp/tmp189jd9a_fstestosfs:
total 4
drwxrwxr-x 3 andrew andrew 4096 Jun 11 02:03 foo

/tmp/tmp1i3v20tjfstestosfs:
total 12
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 egg
drwxrwxr-x 3 andrew andrew 4096 Jun 11 02:03 foo
-rw-rw-r-- 1 andrew andrew   12 Jun 11 02:03 top.txt

/tmp/tmp1l7wo6t1fstestosfs:
total 8
-rw-rw-r-- 1 andrew andrew    3 Jun 11 02:03 file.bin
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 foo

/tmp/tmp396gu0e5fstestosfs:
total 0

/tmp/tmp3em77scofstestosfs:
total 4
-rw-rw-r-- 1 andrew andrew 1200 Jun 11 02:03 hello.bin

/tmp/tmp_4phjqbjfstestosfs:
total 4
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 foo

/tmp/tmp56py379lfstestosfs:
total 4
-rw-rw-r-- 1 andrew andrew 6 Jun 11 02:03 foo

/tmp/tmp5n3vws2vfstestosfs:
total 8
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 foo
-rw-rw-r-- 1 andrew andrew   20 Jun 11 02:03 iter.bin

/tmp/tmp5vgpa86rfstestosfs:
total 4
-rw-rw-r-- 1 andrew andrew 12 Jun 11 02:03 hello.bin

/tmp/tmp5w3s7nnrfstestosfs:
total 8
drwxrwxr-x 3 andrew andrew 4096 Jun 11 02:03 foo
-rw-rw-r-- 1 andrew andrew    4 Jun 11 02:03 foo.bin

/tmp/tmp6n9c6n6nfstestosfs:
total 4
-rw-rw-r-- 1 andrew andrew    0 Jun 11 02:03 bar
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 foo

/tmp/tmp6p8f1wfhfstestosfs:
total 2312
-rw-rw-r-- 1 andrew andrew 1572864 Jun 11 02:03 bar
drwxrwxr-x 2 andrew andrew    4096 Jun 11 02:03 dir1
drwxrwxr-x 3 andrew andrew    4096 Jun 11 02:03 dir2
-rw-rw-r-- 1 andrew andrew  786432 Jun 11 02:03 foo

/tmp/tmp6v96q1refstestosfs:
total 4
drwxrwxr-x 3 andrew andrew 4096 Jun 11 02:03 foo

/tmp/tmp6yliqrgjfstestosfs:
total 0
-rw-rw-r-- 1 andrew andrew 0 Jun 11 02:03 foo

/tmp/tmp7uvz1xbrfstestosfs:
total 2312
-rw-rw-r-- 1 andrew andrew 1572864 Jun 11 02:03 bar
drwxrwxr-x 2 andrew andrew    4096 Jun 11 02:03 dir1
drwxrwxr-x 3 andrew andrew    4096 Jun 11 02:03 dir2
-rw-rw-r-- 1 andrew andrew  786432 Jun 11 02:03 foo

/tmp/tmp8zll3o76fstestosfs:
total 0

/tmp/tmpacu1rkz8fstestosfs:
total 0

/tmp/tmpb3kld447fstestosfs:
total 16
-rw-rw-r-- 1 andrew andrew    4 Jun 11 02:03 bar
-rw-rw-r-- 1 andrew andrew    4 Jun 11 02:03 baz
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 dir
-rw-rw-r-- 1 andrew andrew    4 Jun 11 02:03 foo
-rw-rw-r-- 1 andrew andrew    0 Jun 11 02:03 foo_copy

/tmp/tmpb3qvum7ffstestosfs:
total 2312
-rw-rw-r-- 1 andrew andrew 1572864 Jun 11 02:03 bar
drwxrwxr-x 2 andrew andrew    4096 Jun 11 02:03 dir1
drwxrwxr-x 3 andrew andrew    4096 Jun 11 02:03 dir2
-rw-rw-r-- 1 andrew andrew  786432 Jun 11 02:03 foo

/tmp/tmp_bcvx72hfstestosfs:
total 8
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 egg
drwxrwxr-x 3 andrew andrew 4096 Jun 11 02:03 foo

/tmp/tmpc3r8nk8sfstestosfs:
total 4
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 foo

/tmp/tmpcgud50n1fstestosfs:
total 0

/tmp/tmpck80fzibfstestosfs:
total 0
-rw-rw-r-- 1 andrew andrew 0 Jun 11  2019 birthday.txt

/tmp/tmpcknaa6aifsTempFS:
total 12
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 egg
drwxrwxr-x 3 andrew andrew 4096 Jun 11 02:03 foo
-rw-rw-r-- 1 andrew andrew   12 Jun 11 02:03 top.txt

/tmp/tmpd02eaf4nfstestosfs:
total 0

/tmp/tmpd4p6j_yifstestosfs:
total 16
-rw-rw-r-- 1 andrew andrew    3 Jun 11 02:03 bar
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 baz
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 dir
-rw-rw-r-- 1 andrew andrew    3 Jun 11 02:03 foo
-rw-rw-r-- 1 andrew andrew    0 Jun 11 02:03 notadir

/tmp/tmpda5d2rnrfstestosfs:
total 8
-rw-rw-r-- 1 andrew andrew 12 Jun 11 02:03 bar.txt
-rw-rw-r-- 1 andrew andrew 12 Jun 11 02:03 foo.txt

/tmp/tmpdoz9qly3fstestosfs:
total 4
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 foo

/tmp/tmpffgd9v02fstestosfs:
total 0
-rw-rw-r-- 1 andrew andrew 0 Jun 11 02:03 new.txt

/tmp/tmpfie7smjufsTempFS:
total 0

/tmp/tmpfo71zn9vfstestosfs:
total 16
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 dir
drwxrwxr-x 3 andrew andrew 4096 Jun 11 02:03 foo
-rw-rw-r-- 1 andrew andrew    5 Jun 11 02:03 foo1
-rw-rw-r-- 1 andrew andrew    5 Jun 11 02:03 foo3

/tmp/tmpfu04kylbfstestosfs:
total 0

/tmp/tmpgx1fg97ofsTempFS:
total 12
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 egg
drwxrwxr-x 3 andrew andrew 4096 Jun 11 02:03 foo
-rw-rw-r-- 1 andrew andrew   12 Jun 11 02:03 top.txt

/tmp/tmpgye7zaqafstestosfs:
total 0

/tmp/tmph29y0x71fstestosfs:
total 4
-rw-rw-r-- 1 andrew andrew 3 Jun 11 02:03 foo

/tmp/tmph37u6ndafstestosfs:
total 12
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 egg
drwxrwxr-x 3 andrew andrew 4096 Jun 11 02:03 foo
-rw-rw-r-- 1 andrew andrew   12 Jun 11 02:03 top.txt

/tmp/tmphq76_4bifstestosfs:
total 4
lrwxrwxrwx 1 andrew andrew 30 Jun 11 02:03 bar -> /tmp/tmphq76_4bifstestosfs/foo
-rw-rw-r-- 1 andrew andrew  6 Jun 11 02:03 foo

/tmp/tmpht99mmtvfstestosfs:
total 2312
-rw-rw-r-- 1 andrew andrew 1572864 Jun 11 02:03 bar
drwxrwxr-x 2 andrew andrew    4096 Jun 11 02:03 dir1
drwxrwxr-x 3 andrew andrew    4096 Jun 11 02:03 dir2
-rw-rw-r-- 1 andrew andrew  786432 Jun 11 02:03 foo

/tmp/tmphtnr_i0qfsTempFS:
total 12
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 egg
drwxrwxr-x 3 andrew andrew 4096 Jun 11 02:03 foo
-rw-rw-r-- 1 andrew andrew   12 Jun 11 02:03 top.txt

/tmp/tmphuruhcv0fstestosfs:
total 0

/tmp/tmpip8msjn3fstestosfs:
total 4
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 foo

/tmp/tmpjemvmm05fstestosfs:
total 4
-rw-rw-r-- 1 andrew andrew 7 Jun 11 02:03 test_openbin_exclusive

/tmp/tmpk51k477yfstestosfs:
total 4
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 foo

/tmp/tmpk6fgqg8ofstestosfs:
total 12
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 egg
drwxrwxr-x 3 andrew andrew 4096 Jun 11 02:03 foo
-rw-rw-r-- 1 andrew andrew   12 Jun 11 02:03 top.txt

/tmp/tmpkcoo_7w8fstestosfs:
total 0
-rw-rw-r-- 1 andrew andrew 0 Jun 11 02:03 foo

/tmp/tmpm5g0idjhfstestosfs:
total 8
-rw-rw-r-- 1 andrew andrew 3 Jun 11 02:03 foo1
-rw-rw-r-- 1 andrew andrew 3 Jun 11 02:03 foo2

/tmp/tmpmamlehs3fstestosfs:
total 4
-rw-rw-r-- 1 andrew andrew 384 Jun 11 02:03 foo

/tmp/tmpmm85f801fstestosfs:
total 12
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 egg
drwxrwxr-x 3 andrew andrew 4096 Jun 11 02:03 foo2
-rw-rw-r-- 1 andrew andrew   12 Jun 11 02:03 top.txt

/tmp/tmpmu4ihcz8fstestosfs:
total 0
-rw-rw-r-- 1 andrew andrew 0 Jun 11 02:03 foo

/tmp/tmpn3u8wtfwfstestosfs:
total 0

/tmp/tmpn5bdc9qafstestosfs:
total 2312
-rw-rw-r-- 1 andrew andrew 1572864 Jun 11 02:03 bar
drwxrwxr-x 2 andrew andrew    4096 Jun 11 02:03 dir1
drwxrwxr-x 3 andrew andrew    4096 Jun 11 02:03 dir2
-rw-rw-r-- 1 andrew andrew  786432 Jun 11 02:03 foo

/tmp/tmpnahwvf72fstestosfs:
total 8
drwxrwxr-x 3 andrew andrew 4096 Jun 11 02:03 foo
drwxrwxr-x 3 andrew andrew 4096 Jun 11 02:03 foo2

/tmp/tmpne9qrl70fstestosfs:
total 0

/tmp/tmpnhqb_k6wfsTempFS:
total 4
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 foo

/tmp/tmpnpzk7jdqfstestosfs:
total 4
-rw-rw-r-- 1 andrew andrew 3 Jun 11 02:03 foo

/tmp/tmponstgn0nfstestosfs:
total 2312
-rw-rw-r-- 1 andrew andrew 1572864 Jun 11 02:03 bar
drwxrwxr-x 2 andrew andrew    4096 Jun 11 02:03 dir1
drwxrwxr-x 3 andrew andrew    4096 Jun 11 02:03 dir2
-rw-rw-r-- 1 andrew andrew  786432 Jun 11 02:03 foo

/tmp/tmpooreripefstestosfs:
total 4
-rw-rw-r-- 1 andrew andrew 6 Jun 11 02:03 foo

/tmp/tmpoxv06nh8fstestosfs:
total 8
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 $FOO
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 TYRIONLANISTER

/tmp/tmpp62860iqfstestosfs:
total 8
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 foo
-rw-rw-r-- 1 andrew andrew    0 Jun 11 02:03 fOO
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 Foo

/tmp/tmpps972tg7fstestosfs:
total 2312
-rw-rw-r-- 1 andrew andrew 1572864 Jun 11 02:03 bar
drwxrwxr-x 2 andrew andrew    4096 Jun 11 02:03 dir1
drwxrwxr-x 3 andrew andrew    4096 Jun 11 02:03 dir2
-rw-rw-r-- 1 andrew andrew  786432 Jun 11 02:03 foo

/tmp/tmprre9qmxyfstestosfs:
total 4
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 dir
-rw-rw-r-- 1 andrew andrew    0 Jun 11 02:03 foo

/tmp/tmpruti4s9rfstestosfs:
total 4
-rw-rw-r-- 1 andrew andrew 256 Jun 11 02:03 foo

/tmp/tmpry6y7dsrfstestosfs:
total 16
-rw-rw-r-- 1 andrew andrew    4 Jun 11 02:03 bar
-rw-rw-r-- 1 andrew andrew    4 Jun 11 02:03 baz
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 dir
-rw-rw-r-- 1 andrew andrew    4 Jun 11 02:03 foo

/tmp/tmps0pzvleqfstestosfs:
total 0
-rw-rw-r-- 1 andrew andrew 0 Jul  5  2016 birthday.txt

/tmp/tmpsdp9kws6fstestosfs:
total 4
-rw-rw-r-- 1 andrew andrew    0 Jun 11 02:03 bar
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 dir
-rw-rw-r-- 1 andrew andrew    0 Jun 11 02:03 foo

/tmp/tmpsherufovfstestosfs:
total 4
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 foo

/tmp/tmp_sttbo_3fstestosfs:
total 0

/tmp/tmpux7008cgfstestosfs:
total 12
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 egg
drwxrwxr-x 3 andrew andrew 4096 Jun 11 02:03 foo
-rw-rw-r-- 1 andrew andrew   12 Jun 11 02:03 top.txt

/tmp/tmpu_y2r0xbfstestosfs:
total 4
drwxrwxr-x 3 andrew andrew 4096 Jun 11 02:03 foo

/tmp/tmpvhegnjrffstestosfs:
total 4
drwxrwxr-x 3 andrew andrew 4096 Jun 11 02:03 foo

/tmp/tmpvmsmfb5qfstestosfs:
total 0
-rw-rw-r-- 1 andrew andrew 0 Jun 11 02:03 foo

/tmp/tmpvqz4yw8hfstestosfs:
total 4
-rw-rw-r-- 1 andrew andrew 7 Jun 11 02:03 test_open_exclusive

/tmp/tmpvsv2kvkffstestosfs:
total 4
drwxrwxr-x 3 andrew andrew 4096 Jun 11 02:03 foo
-rw-rw-r-- 1 andrew andrew    0 Jun 11 02:03 test.txt

/tmp/tmpvy9rtio3fstestosfs:
total 8
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 dir
-rw-rw-r-- 1 andrew andrew    3 Jun 11 02:03 foo

/tmp/tmpvzswc9xtfstestosfs:
total 8
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 baz
-rw-rw-r-- 1 andrew andrew  256 Jun 11 02:03 foo

/tmp/tmpwelqzllwfstestosfs:
total 12
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 egg
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 foo
drwxrwxr-x 3 andrew andrew 4096 Jun 11 02:03 foo2

/tmp/tmpwu9dwt0gfstestosfs:
total 8
-rw-rw-r-- 1 andrew andrew    0 Jun 11 02:03 empty
-rw-rw-r-- 1 andrew andrew    1 Jun 11 02:03 one
-rw-rw-r-- 1 andrew andrew 1000 Jun 11 02:03 onethousand

/tmp/tmpx2ophvmffstestosfs:
total 4
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 foo

/tmp/tmpxdu4ss7mfstestosfs:
total 8
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 foo
-rw-rw-r-- 1 andrew andrew   20 Jun 11 02:03 iter.txt

/tmp/tmpxm_kma6rfstestosfs:
total 2312
-rw-rw-r-- 1 andrew andrew 1572864 Jun 11 02:03 bar
drwxrwxr-x 2 andrew andrew    4096 Jun 11 02:03 dir1
drwxrwxr-x 3 andrew andrew    4096 Jun 11 02:03 dir2
-rw-rw-r-- 1 andrew andrew  786432 Jun 11 02:03 foo

/tmp/tmpxrb9ap8r__tempfs__:
total 4
-rw-rw-r-- 1 andrew andrew 3 Jun 11 02:03 foo

/tmp/tmpy04mawiifsTempFS:
total 0

/tmp/tmpy1jzdgtwfsTempFS:
total 4
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 foo

/tmp/tmpy24v7fujfstestosfs:
total 8
-rw-rw-r-- 1 andrew andrew  0 Jun 11 02:03 bin
-rw-rw-r-- 1 andrew andrew 10 Jun 11 02:03 text
-rw-rw-r-- 1 andrew andrew 14 Jun 11 02:03 unicode

/tmp/tmpz4lz_eo2fstestosfs:
total 4
-rw-rw-r-- 1 andrew andrew 3 Jun 11 02:03 foo

/tmp/tmpzb8g0h9sfstestosfs:
total 4
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 bar
-rw-rw-r-- 1 andrew andrew    0 Jun 11 02:03 foo.txt

/tmp/tmpzc7rfkvzfstestosfs:
total 8
-rw-rw-r-- 1 andrew andrew 3 Jun 11 02:03 foo1
-rw-rw-r-- 1 andrew andrew 4 Jun 11 02:03 foo2
-rw-rw-r-- 1 andrew andrew 0 Jun 11 02:03 newfoo

/tmp/tmpzgqa2dvrfstestosfs:
total 12
-rw-rw-r-- 1 andrew andrew    6 Jun 11 02:03 bar
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 baz
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 baz2

/tmp/tmpzu50bz54fstestosfs:
total 8
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 bar
drwxrwxr-x 2 andrew andrew 4096 Jun 11 02:03 baz
-rw-rw-r-- 1 andrew andrew    0 Jun 11 02:03 foo.py
-rw-rw-r-- 1 andrew andrew    0 Jun 11 02:03 foo.pyc
-rw-rw-r-- 1 andrew andrew    0 Jun 11 02:03 foo.txt

Seems like make_fs in https://github.com/PyFilesystem/pyfilesystem2/blob/master/tests/test_osfs.py does tempfile.mkdtemp("fstestosfs") but this directory never gets deleted anywhere :( Was there supposed to be another shutil.rmtree in destroy_fs ?
I guess in some situations you may want to keep these temporary test directories for later debugging purposes, but IMHO running the unit-tests should auto-delete all these junk files by default, and only leave them behind if some kind of env-var or something is explicitly set?

@lurch
Copy link
Contributor Author

lurch commented Jun 11, 2019

Was there supposed to be another shutil.rmtree in destroy_fs ?

That's part of the problem. Another part of the problem is that fs/test.py makes a lot of open_fs calls outside of a with context-handler, so all the TempFS instances that get opened never get explicitly close()d, and so their tempdirs don't get deleted. (I've made a local fix for this, which seems to help matters significantly)

Would it also make sense for FSes to be automatically close()d when their object gets garbage-collected, or is not doing that a deliberate part of the PyFilesystem design / interface?

@willmcgugan
Copy link
Member

Agree that the temp directories should be cleaned up. I don't recall ever looking at them to debug, so that's no great loss.

I've found that code in __del__ to be a constant source of pain, which is probably why I avoided it this time around. It broke garbage collection, and it was hard to know which attributes could be relied on in the destructor.

That said, it is a useful pattern. And clearly I do use open_fs without a with statement.

I think Python3.4 made some major improvements regarding destructors that made them more reliable, and thats our minimum Python3 version. I don't intend to support Python2.7 for much longer. So perhaps adding an auto-closing __del__ to FS base would be a good move now.

@willmcgugan willmcgugan self-assigned this Jun 11, 2019
@willmcgugan
Copy link
Member

Fixed in v2.4.9

@dargueta
Copy link
Contributor

dargueta commented Jul 29, 2019

So I know this is a bit late but I just saw this issue and I think doing the fix in __del__ is probably not the best way to go about this. For archive file systems, an exception thrown somewhere inside the file system could indicate corruption or some other problem and we don't want to finalize changes. Same thing with anything transaction-based.

Instead, I suggest we should use tempfile.TemporaryDirectory in the tests instead of mkdtemp. Python 2 doesn't have it, but there's a backport. To preserve the behavior we could add a call to close() in the __del__ of TempFS.

@willmcgugan
Copy link
Member

I don't think we can avoid the close on exit. If you use the filesystem as a context manager, which is the recommended way, then it's going to call close(), exception or not.

I can see the use case for abandoning writing an archive if there is an error. I think it is is peculiar to archives.

How about adding a parameter to close(), which would allow you to close the filesystem without writing the archive. Something like the following:

with open_fs("zip://foo.zip", create=True) as zip_fs:
    try:
        write_files_to_zip(zip_fs)
    except:
        zip_fs.close(save=False)

This would actually call close() twice if there is an exception, but the second call would be a null-op.

@althonos may be interested in this conversation.

@dargueta
Copy link
Contributor

dargueta commented Jul 29, 2019

Your example would put the onus on the user to enforce the safe behavior, but I think you're onto something. I'm tinkering with an implementation of some file systems (FAT, ext2, etc.) so people can read/write disk images. They essentially behave very similar to archives, so I have a vested interest in this. 🙂

@lurch
Copy link
Contributor Author

lurch commented Jul 29, 2019

Your example would put the onus on the user to enforce the safe behavior, but I think you're onto something.

Maybe this is another attribute that https://pyfilesystem2.readthedocs.io/en/latest/reference/base.html#fs.base.FS.getmeta could be used for?

@willmcgugan
Copy link
Member

@dargueta I'm not sure there is a safe behavior per se. A user may still want to write the archive with the files up until the failure.

@lurch What do you suggest?

@lurch
Copy link
Contributor Author

lurch commented Jul 29, 2019

@willmcgugan I'm not sure what I'm suggesting, just throwing out random ideas ;-) Maybe an 'only _updates_on_close' or something? 🤷‍♂️

A user may still want to write the archive with the files up until the failure.

Hmm, in that case maybe it's worth making it an init-param, rather than a close-param (IYSWIM) ? Seems likely to me that you'd want all places where 'close' is called on an FS to have consistent behaviour?

EDIT: ignore my "maybe it's worth making it an init-param, rather than a close-param" comment - I made it while tired and I now realise that the close method has no idea whether it's being called in an exception-handler or not, so the only sensible way of doing this is the try...except that Will suggested.

@lurch
Copy link
Contributor Author

lurch commented Jul 30, 2019

IMHO this issue should be re-opened. I've just re-run the commands from my initial comment in this thread on the latest master, and while #319 has eliminated all the leftover tempfs directories, there's still an awful lot of leftover osfs directories 🙁

@dargueta the "don't close an FS after an exception" discussion should probably be moved to a separate issue?

@willmcgugan willmcgugan reopened this Jul 30, 2019
@dargueta
Copy link
Contributor

the "don't close an FS after an exception" discussion should probably be moved to a separate issue?

Will do

@willmcgugan
Copy link
Member

Pushed some changes to master which should delete the remaining tmp files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants