Modify

Ticket #171 (closed task: invalid)

Opened 6 years ago

Last modified 5 years ago

large time on wa (waiting io)

Reported by: yurj Owned by: andycat
Priority: major Milestone: 0.2.4
Component: ATVideo Severity:
Keywords: Cc: yurj@…
Who will test this:

Description

Hi!

using plumi 0.2:

more /var/lib/plone25/zeocluster/Products/ATVideo/version.txt 0.2 build 17 more /var/lib/plone25/zeocluster/Products/ExternalStorage/version.txt 0.7 (SVN/Unreleased) more /var/lib/plone25/zeocluster/Products/ATMediaFile/version.txt 0.1 build 66

it is very slow on shoing things. Using top, I can see the cpu 100% on wa (waiting io).

Can this be a problem of externalstorage?

Attachments

Change History

comment:1 Changed 6 years ago by anonymous

can be SearchableText?? I've uploaded 1 GB of videos, and reindexing SearchableText? take 30 seconds. Archetypes does a lot of reindexing, can be the guilt?

comment:2 Changed 6 years ago by yurj

class VideoView?( BrowserView? ):

u"""This browser view is used as utility for the atvideo view """ implements( IVideoView, ITopicsProvider )

def init(self, context, request):

super(VideoView?, self).init(context, request) self.portal_url = getToolByName(self.context, "portal_url")() self.vocab_tool = getToolByName(self.context, "portal_vocabularies") self.use_vpip = "vpip" in context.Subject()

media_info = context.getFileAttribs() self.enclosure = media_info[1] > 0

(self.transcoding_ready,

self.transcoding_status) = self.get_transcoding_status()

is this ok? it is writing every time I see an item.

Also getFileAttribs is called two or three times on every view:


2008-07-24T17:50:40 INFO getFileAttribs using the CACHE of file system , length 6342048


2008-07-24T17:50:43 INFO getFileAttribs using the CACHE of file system , length 6342048


2008-07-24T17:50:45 INFO getFileAttribs using the CACHE of file system , length 6342048

ATVideo/skins/ATEngageVideo/video_view.pt: tal:define="enc here/getFileAttribs" ATVideo/skins/ATEngageVideo/video_view.pt: tal:define="enc here/getFileAttribs ; bt_url here/getTorrentURL" ATVideo/browser/video.py: media_info = context.getFileAttribs()

comment:3 Changed 6 years ago by monossido

Seems i have the same issue, with a 400mb of video the page takes a lot of seconds to appear and 100% of cpu Here http://plumi.org/ticket/102#comment:3 i read "yes, this is a known issue with large file support inside Plone.

You should upgraded to Plone 2.5.4-2"

anyone have test this?

i'm italian so sorry for my bad english ;)

comment:4 Changed 6 years ago by yurj

no, it does'nt change.

Maybe this is the real problem!

 http://dev.plone.org/plone/ticket/7586

"""FileField?.getFilename by default calls getBaseUnit to get the filename. getBaseUnit recalls FileField?.getFilename with fromBaseUnit to false to get the filename ! After, it creates an BaseUnit? and loads the file content into memory (value.data). With big files, it's very slow and consummes a lot of RAM for nothing. After all this work, BaseUnit?.getFilename is called.

This is very complex and unefficient ! Why FileField?.getFilename needs to create a baseUnit only to get a filename that is got finally from itself ? """

comment:5 Changed 6 years ago by andycat

  • Milestone set to 0.3 - Plone 3

I might close this ticket *after* we move to Blob fields, and if we dont get any more comments/feedback.

comment:6 Changed 6 years ago by yurj

it has nothing to do with the storage. It is the standard template loading data from the content in the wrong way, just take a look to it with a big file.

takes 5 minute to test it.

comment:7 Changed 6 years ago by andycat

narky comments about the length of time something takes to test arent helpful. Fact is : this bug report isnt very good quality, its very hard to reproduce whatever problem it is you are trying to describe.

comment:8 Changed 6 years ago by anonymous

good quality:

install Plumi (0.2 oe 0.3, no differences about this topic) upload a 200MB video, wait the transcoding in flash view the page with the video (watch the event.log with tail -f to see messages about sizes and caching it)

It takes a lot of seconds to just render the view (doing the same thing three time, as you can see in the event.log)

Try it, takes few minutes.

comment:9 Changed 6 years ago by yurj

simpler, go to:  http://studidirittopenale.cab.unipd.it/ and try to see some videos.

comment:10 Changed 6 years ago by andycat

So a video view is taking longer that you like ? How slow do you define too slow? Have you cached your site using apache2/squid ?

You say "It is the standard template loading data from the content in the wrong way, just take a look to it with a big file."

What is the template doing wrong?

As my comment above was alluding, once we move into Plone 3 and ATBlob fields, the speed of ZODB operations should be quicker, since the large binary objects (video) will be handled better.

comment:11 Changed 6 years ago by yurj

Yes, I've cached but that content (the video object) send a nice (...) "please, don't cache me", if you look at the headers you'll see it. Even CacheFu? can't override this behaviour, I can't find where to look deeper...

(i'm using apache cache on the virtual host:

CacheRoot? /var/cache/www/dirittopenale CacheEnable? disk /

)

If you see, the home page load fast, being it cached by apache.

Anyway, the storage will not change anything, somewhere the video is loaded on memory or the code is doing some operation that load the video in memory, maybe deep in Plone or Archetypes or Zope.

I just ask you to try to see, quickly, why it takes so long. The flash part should not take long, it is just text. I suppose it is some code calculating the video size, and as side effect, loading it (or most) in memory.

You can't see it with small videos, you've to load a about 100 MB file (and more) to see it clearly.

I'm happy with Plumi, this is the only real issue I've found :)

comment:12 Changed 5 years ago by and

This should be solved by Plumi alpha. Yurj is this fixed for you in Plumi alpha?

comment:13 Changed 5 years ago by and

sorry, plumi 03.-alpha

comment:14 Changed 5 years ago by vik

  • Milestone changed from 3.1 to 0.2.4

comment:15 Changed 5 years ago by yurj

I'm glad to see the fix for 0.2.4 too :)

comment:16 Changed 5 years ago by and

  • Status changed from new to closed
  • Resolution set to invalid

ok, this should be solved with 3.0 and as there is now no 0.2.4 release that is going to happen - just a small updated to 0.2.3 to enable migration - am closing this as invalid. please re-open if relevant.

View

Add a comment

Modify Ticket

Action
as closed
The resolution will be deleted. Next status will be 'reopened'
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.