Page MenuHomePhabricator

importImages.php broken on cluster
Closed, ResolvedPublic

Description

reedy@tin:/tmp/uploads$ sudo -u apache mwscript importImages.php --wiki=commonswiki --user="Jdforrester_(WMF)" /tmp/uploads
Import Images

Importing linux.conf.au 2014 - VisualEditor presentation.ogv...PHP Warning: fopen(/tmp/uploads/linux.conf.au 2014 - VisualEditor presentation.ogv): failed to open stream: Permission denied in /a/common/php-1.23wmf10/includes/filebackend/SwiftFileBackend.php on line 278

Warning: fopen(/tmp/uploads/linux.conf.au 2014 - VisualEditor presentation.ogv): failed to open stream: Permission denied in /a/common/php-1.23wmf10/includes/filebackend/SwiftFileBackend.php on line 278
done.

Found: 1
Added: 1
reedy@tin:/tmp/uploads$

The added is a lie. The upload didn't complete, it's a 404

https://upload.wikimedia.org/wikipedia/commons/e/ea/Linux.conf.au_2014_-_VisualEditor_presentation.ogv

404 Not Found

The resource could not be found.

File not found: /v1/AUTH_mw/wikipedia-commons-local-public.ea/e/ea/Linux.conf.au_2014_-_VisualEditor_presentation.ogv


Version: wmf-deployment
Severity: normal
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=60777

Details

Reference
bz60318

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 2:54 AM
bzimport set Reference to bz60318.

If I chmod 777 it, I can upload it :/

I don't see any way around getting errors when trying to upload files as a user that can't see them. What exactly is this about?

(In reply to comment #2)

I don't see any way around getting errors when trying to upload files as a
user
that can't see them. What exactly is this about?

The behaviour changed at some point (I didn't have to change file permissions before). But even with read for everyone on /tmp/uploads, it doesn't work with errors.

There is a related error that the uploads apparently succeed, even though they weren't imported.

reedy@tin:/tmp/uploads$ sudo -u apache mwscript importImages.php --wiki=commonswiki --user="Odder" /tmp/uploads
Import Images

Importing 2014 State Of The Union Address Enhanced.webm...PHP Warning: fopen(/tmp/uploads/2014 State Of The Union Address Enhanced.webm): failed to open stream: Permission denied in /a/common/php-1.23wmf11/includes/filebackend/SwiftFileBackend.php on line 278

Warning: fopen(/tmp/uploads/2014 State Of The Union Address Enhanced.webm): failed to open stream: Permission denied in /a/common/php-1.23wmf11/includes/filebackend/SwiftFileBackend.php on line 278
done.
Importing 2013-10-05 President Obama's Weekly Address.webm...PHP Warning: fopen(/tmp/uploads/2013-10-05 President Obama's Weekly Address.webm): failed to open stream: Permission denied in /a/common/php-1.23wmf11/includes/filebackend/SwiftFileBackend.php on line 278

Warning: fopen(/tmp/uploads/2013-10-05 President Obama's Weekly Address.webm): failed to open stream: Permission denied in /a/common/php-1.23wmf11/includes/filebackend/SwiftFileBackend.php on line 278
done.
Importing 2013-10-12 President Obama's Weekly Address.webm...PHP Warning: fopen(/tmp/uploads/2013-10-12 President Obama's Weekly Address.webm): failed to open stream: Permission denied in /a/common/php-1.23wmf11/includes/filebackend/SwiftFileBackend.php on line 278

Warning: fopen(/tmp/uploads/2013-10-12 President Obama's Weekly Address.webm): failed to open stream: Permission denied in /a/common/php-1.23wmf11/includes/filebackend/SwiftFileBackend.php on line 278
done.

Found: 3
Added: 3
reedy@tin:/tmp/uploads$ ls -al
total rOMWC24047921fcbc
drwxrwxrwx 2 reedy wikidev 4096 Feb 3 18:47 .
drwxrwxrwt 46 root root 12288 Feb 3 18:46 ..
-rw-rw-r-- 1 reedy wikidev 1255 Feb 1 14:50 2013-10-05 President Obama's Weekly Address.txt
-rw-rw-r-- 1 reedy wikidev 163504321 Jan 27 22:13 2013-10-05 President Obama's Weekly Address.webm
-rw-rw-r-- 1 reedy wikidev 1197 Feb 1 14:52 2013-10-12 President Obama's Weekly Address.txt
-rw-rw-r-- 1 reedy wikidev 158637780 Feb 1 14:34 2013-10-12 President Obama's Weekly Address.webm
-rw-rw-r-- 1 reedy wikidev 884 Jan 31 16:26 2014 State Of The Union Address Enhanced.txt
-rw-rw-r-- 1 reedy wikidev 909068945 Jan 30 19:37 2014 State Of The Union Address Enhanced.webm
-rw-r--r-- 1 reedy wikidev 909060459 Feb 3 18:45 SOTU.tar.gz
-rw-r--r-- 1 reedy wikidev 322185685 Feb 3 18:46 WeeklyAddresses.tar.gz
reedy@tin:/tmp/uploads$
reedy@tin:/tmp/uploads$
reedy@tin:/tmp/uploads$
reedy@tin:/tmp/uploads$
reedy@tin:/tmp/uploads$
reedy@tin:/tmp/uploads$ chmod 777 *
reedy@tin:/tmp/uploads$ rm *.gz
reedy@tin:/tmp/uploads$ ls -al
total 1202392
drwxrwxrwx 2 reedy wikidev 4096 Feb 3 18:52 .
drwxrwxrwt 46 root root 12288 Feb 3 18:50 ..
-rwxrwxrwx 1 reedy wikidev 1255 Feb 1 14:50 2013-10-05 President Obama's Weekly Address.txt
-rwxrwxrwx 1 reedy wikidev 163504321 Jan 27 22:13 2013-10-05 President Obama's Weekly Address.webm
-rwxrwxrwx 1 reedy wikidev 1197 Feb 1 14:52 2013-10-12 President Obama's Weekly Address.txt
-rwxrwxrwx 1 reedy wikidev 158637780 Feb 1 14:34 2013-10-12 President Obama's Weekly Address.webm
-rwxrwxrwx 1 reedy wikidev 884 Jan 31 16:26 2014 State Of The Union Address Enhanced.txt
-rwxrwxrwx 1 reedy wikidev 909068945 Jan 30 19:37 2014 State Of The Union Address Enhanced.webm
reedy@tin:/tmp/uploads$ sudo -u apache mwscript importImages.php --wiki=commonswiki --user="Odder" /tmp/uploads
Import Images

Importing 2014 State Of The Union Address Enhanced.webm...done.
Importing 2013-10-05 President Obama's Weekly Address.webm...done.
Importing 2013-10-12 President Obama's Weekly Address.webm...done.

Found: 3
Added: 3
reedy@tin:/tmp/uploads$

Change 110993 had a related patch set uploaded by Aaron Schulz:
Use "rb" instead of "rb+" for Swift store operation handles

https://gerrit.wikimedia.org/r/110993

Change 111111 had a related patch set uploaded by Aaron Schulz:
Fixes and cleanups to FileOpBatch

https://gerrit.wikimedia.org/r/111111

Change 111111 merged by jenkins-bot:
Fixes and cleanups to FileOpBatch

https://gerrit.wikimedia.org/r/111111

Change 111115 had a related patch set uploaded by Aaron Schulz:
Use "rb" instead of "rb+" for Swift store operation handles

https://gerrit.wikimedia.org/r/111115

Change 111116 had a related patch set uploaded by Aaron Schulz:
Fixes and cleanups to FileOpBatch

https://gerrit.wikimedia.org/r/111116

Change 110993 merged by jenkins-bot:
Use "rb" instead of "rb+" for Swift store operation handles

https://gerrit.wikimedia.org/r/110993

Change 111115 merged by jenkins-bot:
Use "rb" instead of "rb+" for Swift store operation handles

https://gerrit.wikimedia.org/r/111115

Change 111116 merged by Aaron Schulz:
Fixes and cleanups to FileOpBatch

https://gerrit.wikimedia.org/r/111116

Gilles raised the priority of this task from High to Unbreak Now!.Dec 4 2014, 10:11 AM
Gilles moved this task from Untriaged to Done on the Multimedia board.
Gilles lowered the priority of this task from Unbreak Now! to High.Dec 4 2014, 11:21 AM