Page MenuHomePhabricator

WindowsError: [Error 32] while renaming log file
Closed, ResolvedPublic

Description

Getting [[Kategorie:Wikipedia:Seite mit fehlendem References-Tag]] list...
Traceback (most recent call last):

File "C:\Python27\lib\logging\handlers.py", line 78, in emit
  self.doRollover()
File "C:\Python27\lib\logging\handlers.py", line 338, in doRollover
  os.rename(self.baseFilename, dfn)

WindowsError: [Error 32] Der Prozess kann nicht auf die Datei zugreifen, da sie
von einem anderen Prozess verwendet wird
Logged from file wikipedia.py, line 9333
Getting 5 pages from wikipedia:de...
Traceback (most recent call last):

File "C:\Python27\lib\logging\handlers.py", line 78, in emit
  self.doRollover()
File "C:\Python27\lib\logging\handlers.py", line 338, in doRollover
  os.rename(self.baseFilename, dfn)

WindowsError: [Error 32] Der Prozess kann nicht auf die Datei zugreifen, da sie
von einem anderen Prozess verwendet wird
Logged from file wikipedia.py, line 9333

Briefwahl <<<

Traceback (most recent call last):

File "C:\Python27\lib\logging\handlers.py", line 78, in emit
  self.doRollover()
File "C:\Python27\lib\logging\handlers.py", line 338, in doRollover
  os.rename(self.baseFilename, dfn)

WindowsError: [Error 32] Der Prozess kann nicht auf die Datei zugreifen, da sie
von einem anderen Prozess verwendet wird
Logged from file wikipedia.py, line 9333
No changes necessary: references tag found.
Traceback (most recent call last):

File "C:\Python27\lib\logging\handlers.py", line 78, in emit
  self.doRollover()
File "C:\Python27\lib\logging\handlers.py", line 338, in doRollover
  os.rename(self.baseFilename, dfn)

WindowsError: [Error 32] Der Prozess kann nicht auf die Datei zugreifen, da sie
von einem anderen Prozess verwendet wird
Logged from file wikipedia.py, line 9333

C:\pwb\compat>version.py
Pywikipedia wikipedia.py (r-1 (unknown), ???????, 2013/09/19, 07:37:28, OUTDATED
)
Python 2.7.3 (default, Apr 10 2012, 23:24:47) [MSC v.1500 64 bit (AMD64)]
config-settings:
use_api = True
use_api_login = True
unicode test: ok

C:\pwb\compat>


Version: unspecified
Severity: normal

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 2:20 AM
bzimport set Reference to bz54685.
bzimport added a subscriber: Unknown Object (????).

btw line 9333 is:

log.log(_level, text, extra=context, **kwargs)

Does this also happen on core, on Windows?

Yes it happens again in core. See also T93357

Change 692617 had a related patch set uploaded (by Xqt; author: Xqt):

[pywikibot/core@master] [IMPR] re-use throttle PID (Step 1)

https://gerrit.wikimedia.org/r/692617

Change 692624 had a related patch set uploaded (by Xqt; author: Xqt):

[pywikibot/core@master] [IMPR] Use different logfiles for multiple processes (Step 2)

https://gerrit.wikimedia.org/r/692624

Change 692617 merged by jenkins-bot:

[pywikibot/core@master] [IMPR] re-use throttle PID (Step 1)

https://gerrit.wikimedia.org/r/692617

Change 692624 merged by jenkins-bot:

[pywikibot/core@master] [IMPR] Use different logfiles for multiple processes (Step 2)

https://gerrit.wikimedia.org/r/692624

JJMC89 reopened this task as Open.EditedMay 26 2021, 2:52 AM
JJMC89 subscribed.

{d0ae0ff3bded6b271557dccf88dd5c96ab7dada3} splits logs for the same script across multiple files even when multiple copies of that script aren't being run at the same time.

Example
$ python t1.py -log  # pid: 1; logs to t1-bot.log; OK
$ # t1.py ends
$ python t2.py -log  # pid: 1; logs to t2-bot.log; OK
$ python t1.py -log  # pid: 2; logs to t1-2-bot.log; should be t1-bot.log
$ # t1.py and t2.py end
$ python t1.py -log  # pid: 1; logs to t1-bot.log; OK

This makes it difficult to read the log chronologically across multiple runs when other scripts may run at overlapping times.

The example is a simple case. Adding more simultaneously running scripts fragments logs further.

Change 695200 had a related patch set uploaded (by Xqt; author: Xqt):

[pywikibot/core@master] [IMPR] Use different logfiles for multiple processes (Step 2) - Add a hash of the script name to throttle log. - Add a new method "get_pid" to Throttle which which returns the pid (process identifier) if there a script is running multiple times; return 0 otherwise. - Deprecate multiplydelay parameter

https://gerrit.wikimedia.org/r/695200

Change 695200 merged by jenkins-bot:

[pywikibot/core@master] [IMPR] Use different logfiles for multiple processes (Step 2)

https://gerrit.wikimedia.org/r/695200