Announcement

Collapse
No announcement yet.

A bit of server tweaking please.

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    A bit of server tweaking please.

    Hello Willow and others,

    Now that the files have grown to greater and greater sizes it might be time to review the server cues and time issues. I started a D/L of 8 files early this morning and then went to bed. When I got up I found that the server sent all but about 2MB of each of the files. That requires that I re-D/L the same files and use a bunch more bandwidth. I have no idea if its time to expand the server cluster or simply to recalibrate to allow for the more common larger files but it is time to do something. This is only one of the reasons that I cancelled my renewal after this 6 mo period is up but its one of the more important ones.

    Good Luck!

    #2
    Sounds like you need a down load manager.
    See this in FQA http://www.abbywinters.com/main.php?page=wiki&id=HowDlm
    Here the link to a good manager http://www.dc-sakura.com/
    This works with fire fox you just have to add the plug-in
    as well as IE 6
    c

    Comment


      #3
      Hi Numlock,

      bandwidth issues are a tricky thing. On the one hand, a fatter pipe will mean higher operating costs which will end up in higher subscription fees. On the other hand, the problems you are experiencing is not necessarily due to anything at our end. In fact, I would bank on it. If you kicked off 8 downloads and they all aborted before hitting the end, then that sounds like you got a dropped connection which is more likely to be down to your ISP, but can in fact happen at any point in the chain between your computer and our servers.

      As Chasaffton says, a download manager is vital if you're doing overnight downloads. These are designed to cope with intermittent connection failures and if a transfer craps out part way through, they will pick up from where they left off. Our servers are configured and tested to work with most popular download managers.

      Comment


        #4
        Originally posted by willow
        These are designed to cope with intermittent connection failures and if a transfer craps out part way through, they will pick up from where they left off.
        Usually but not always...

        Your WMV files are marked as non resumable though in practice they are resumable, go figure
        If your download manager is a bit on the dopey side, as the file is flagged as non resumable it will dump a partial downloaded file and start a new copy. The better download manager will make the attempt to resume the partial download anyway and the file transfer will continue from where it left off.

        Comment


          #5
          Originally posted by ThePope
          Your WMV files are marked as non resumable though in practice they are resumable, go figure
          Yeah, so you've said before, but everything looks fine at our end. what browser/dl manager are you seeing this with?
          Originally posted by ThePope
          If your download manager is a bit on the dopey side, as the file is flagged as non resumable it will dump a partial downloaded file and start a new copy.
          That's what you get for using a dopey download manager In my experience half-assed dl managers (and there's plenty out there) never work properly no matter what.

          Comment


            #6
            Originally posted by willow
            Yeah, so you've said before, but everything looks fine at our end. what browser/dl manager are you seeing this with?
            Tried out quite a few, they all show wmv files as non resumable and file size info is missing.
            Net Transport
            LeechGet
            Free Download Manager (dopey one)
            NetAnts (seriously dopey, wont even begin a download due to unknown file size)

            Originally posted by willow
            That's what you get for using a dopey download manager In my experience half-assed dl managers (and there's plenty out there) never work properly no matter what.
            LOL too true
            As I said before, it's not a problem at this end. I use Net Transport which works well.
            Was just presenting the info for the benefit of others, why everything is fine at your end is a bit odd though.

            Comment


              #7
              Looking at the HTTP headers when doing a download there's no Content-Length information available for wmv files, with both mpg and zip files that information is available. I guess that's why they're showing as non resumable.

              If your interested I'll post the header logs.

              Comment


                #8
                In this instance I was trying to D/L the four Jacki & Armine videos as well as the four Armine videos. This would normally take about an hour with all eight downloads running at the same time. I tested a theory and opened connections to only the Armine videos and they came across just fine. I did the same with the larger sets for Jacki & Armine as well as the extra large picture set for Dee H (5 D/Ls in total) and the downloads timed out. It is possible to repeat this situation so I know that it is not a 'fluke' with my ISP. If I had to guess I'd say that its something to do with your load balancing software/hardware (We HOPE that your load balancing is hardware driven!) I also tested a theory and opened a 2Gb D/L from another site in Milpitas Ca that's only a single hop different than AW. That D/L transferred just fine and continued even after the AW downloads timed out. This again points to something on the AW server side of things. I have never required a download manager because Firefox handles things so nicely. I leave D/L managers to those who run MSIE or other less robust browsers.

                Anyway, as I said before when my subscription runs out in January that's it for me and AW for now. I've enjoyed the content and the presentation for several years now. You've done a fine job at making MANY people happy with what you have to offer here on the AW site. Keep up the good work but I'm sure that I'm not the only person who will benefit from a bit of research to discover the cause of dropped connections.

                Best Regards

                Comment


                  #9
                  Originally posted by ThePope
                  Looking at the HTTP headers when doing a download there's no Content-Length information available for wmv files, with both mpg and zip files that information is available. I guess that's why they're showing as non resumable.

                  If your interested I'll post the header logs.
                  All the headers and resuming looks good to me!

                  badl@vm32 badl $ wget --header "Cookie: user=xxxxxxxxxxxxxxxxxxxxxxxxxx" http://media.abbywinters.com/member/alintavid1.wmv
                  --23:55:30-- http://media.abbywinters.com/member/alintavid1.wmv
                  => `alintavid1.wmv'
                  Resolving media.abbywinters.com... 66.240.207.190
                  Connecting to media.abbywinters.com[66.240.207.190]:80... connected.
                  HTTP request sent, awaiting response... 200 OK
                  Length: 69,080,295 [video/x-ms-wmv]

                  9% [===============> ] 6,466,453 325.72K/s ETA 03:26
                  badl@vm32 badl $ wget -c --header "Cookie: user=xxxxxxxxxxxxxxxxxxxxxxxxxx" http://media.abbywinters.com/member/alintavid1.wmv
                  --23:56:20-- http://media.abbywinters.com/member/alintavid1.wmv
                  => `alintavid1.wmv'
                  Resolving media.abbywinters.com... 66.240.207.190
                  Connecting to media.abbywinters.com[66.240.207.190]:80... connected.
                  HTTP request sent, awaiting response... 206 Partial Content
                  Length: 69,080,295 (62,550,130 to go) [video/x-ms-wmv]

                  19% [++++++++++++++++=================> ] 13,581,550 382.16K/s ETA 02:30
                  badl@vm32 badl $

                  Comment


                    #10
                    Wonder what the hell is going on here then, guess it's either my system or ISP that's the cause. Why wmv file data is being delivered differently at my end is really strange. Just tried with a firewall disabled connection, same result. Doing a direct download with Firefox is the same also.

                    Here's a log for both mpg and wmv from NetAnts which is absolute crap, it wont even attempt a download when file size or Content-Length is unknown.

                    WMV File:
                    2005/11/02 09:09:27 Resolve host address ...
                    2005/11/02 09:09:27 Host address resolved
                    2005/11/02 09:09:27 Connect to host (media.abbywinters.com:80) ...
                    2005/11/02 09:09:27 Connect to host successfully
                    2005/11/02 09:09:27 GET /member/alintavid1.wmv HTTP/1.1
                    2005/11/02 09:09:27 Host: media.abbywinters.com
                    2005/11/02 09:09:27 Accept: */*
                    2005/11/02 09:09:27 User-Agent: NetAnts/1.25
                    2005/11/02 09:09:27 Pragma: no-cache
                    2005/11/02 09:09:27 Cache-Control: no-cache
                    2005/11/02 09:09:27 Connection: close
                    2005/11/02 09:09:27 Cookie: user=ThePope%3A%3A%3**************
                    2005/11/02 09:09:28 HTTP/1.0 200 OK
                    2005/11/02 09:09:28 Connection: close
                    2005/11/02 09:09:28 Date: Tue, 01 Nov 2005 04:07:56 GMT
                    2005/11/02 09:09:28 Server: Apache/2.0.52 (Unix) DAV/2 PHP/5.0.5
                    2005/11/02 09:09:28 Last-Modified: Mon, 31 Oct 2005 06:27:08 GMT
                    2005/11/02 09:09:28 ETag: "80f9-41e14e7-edb3c700"
                    2005/11/02 09:09:28 Accept-Ranges: bytes
                    2005/11/02 09:09:28 Content-Type: video/x-ms-wmv
                    2005/11/02 09:09:28 The actual size of downloading file is unknown
                    ================================================== ================
                    MPG File:
                    2005/11/02 09:08:00 Resolve host address ...
                    2005/11/02 09:08:00 Host address resolved
                    2005/11/02 09:08:00 Connect to host (media.abbywinters.com:80) ...
                    2005/11/02 09:08:00 Connect to host successfully
                    2005/11/02 09:08:00 GET /member/alintavid1.mpg HTTP/1.1
                    2005/11/02 09:08:00 Host: media.abbywinters.com
                    2005/11/02 09:08:00 Accept: */*
                    2005/11/02 09:08:00 User-Agent: NetAnts/1.25
                    2005/11/02 09:08:00 Pragma: no-cache
                    2005/11/02 09:08:00 Cache-Control: no-cache
                    2005/11/02 09:08:00 Connection: close
                    2005/11/02 09:08:00 Cookie: user=ThePope%3A%3A%3**************
                    2005/11/02 09:08:00 HTTP/1.0 200 OK
                    2005/11/02 09:08:00 Connection: close
                    2005/11/02 09:08:00 Date: Tue, 01 Nov 2005 22:09:21 GMT
                    2005/11/02 09:08:00 Server: Apache/2.0.52 (Unix) DAV/2 PHP/5.0.5
                    2005/11/02 09:08:00 Last-Modified: Mon, 31 Oct 2005 06:24:37 GMT
                    2005/11/02 09:08:00 ETag: "100b5-4119804-e4b3b340"
                    2005/11/02 09:08:00 Accept-Ranges: bytes
                    2005/11/02 09:08:00 Content-Length: 68261892
                    2005/11/02 09:08:00 Content-Type: video/mpeg
                    2005/11/02 09:08:00 Receiving data...
                    Attached Files

                    Comment


                      #11
                      without being a techie my downlaods seem to go along alright ,at times I can download 2 files at a time and they seem to be all there .I use msie and it seems to do the job at my location and connection .

                      Comment


                        #12
                        Headers all look good for me in firefox. As you can see from the header dialogs below, the content-length and accept-ranges headers are both present and correct.

                        For WMV:
                        Code:
                        GET /member/alintavid1.wmv HTTP/1.1
                        Host: media.abbywinters.com
                        User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7
                        Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
                        Accept-Language: en-us,en;q=0.5
                        Accept-Encoding: gzip,deflate
                        Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
                        Keep-Alive: 300
                        Connection: keep-alive
                        Cookie: user=xxxxxxxxxx; sessiondata=xxxxxxxx; session=6918
                        
                        HTTP/1.x 200 OK
                        Date: Tue, 01 Nov 2005 23:41:30 GMT
                        Server: Apache/2.0.52 (Unix) DAV/2 PHP/5.0.5
                        Last-Modified: Mon, 31 Oct 2005 06:27:08 GMT
                        Etag: "100f9-41e14e7-edb3c700"
                        Accept-Ranges: bytes
                        Content-Length: 69080295
                        Keep-Alive: timeout=15, max=100
                        Connection: Keep-Alive
                        Content-Type: video/x-ms-wmv
                        For MPG:
                        Code:
                        GET /member/alintavid1.mpg HTTP/1.1
                        Host: media.abbywinters.com
                        User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7
                        Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
                        Accept-Language: en-us,en;q=0.5
                        Accept-Encoding: gzip,deflate
                        Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
                        Keep-Alive: 300
                        Connection: keep-alive
                        Cookie: user=xxxxxxxxxxxx; sessiondata=xxxxxxxxxxxx; session=6918
                        
                        HTTP/1.x 200 OK
                        Date: Tue, 01 Nov 2005 23:42:17 GMT
                        Server: Apache/2.0.52 (Unix) DAV/2 PHP/5.0.5
                        Last-Modified: Mon, 31 Oct 2005 06:24:37 GMT
                        Etag: "100b5-4119804-e4b3b340"
                        Accept-Ranges: bytes
                        Content-Length: 68261892
                        Keep-Alive: timeout=15, max=99
                        Connection: Keep-Alive
                        Content-Type: video/mpeg

                        Comment


                          #13
                          yeah we know it's all good for you, question at hand is why is this not so for me.
                          or is this just another of lifes little mysteries I need to accept

                          Comment


                            #14
                            Originally posted by ThePope
                            or is this just another of lifes little mysteries
                            I think this is actually one of life's BIG mysteries, and should be right up there alongside the Loch Ness Monster, and how George W Bush got a second term as Prez.

                            Comment


                              #15
                              Bush is no mystery, surely you heard he's the chosen one. The man is on a mission from god no less.
                              Sorry can't help you with nessie...

                              Comment


                                #16
                                funny stuff he he

                                Comment


                                  #17
                                  solved...
                                  turns out the missing Content-Length info is due to an incorrectly configured proxy server at my ISP's end.

                                  Comment

                                  Subscribe to our e-mail newsletter

                                   
                                  Sign up for the abby newsletter. Don't worry, we'll NEVER share your email address with anyone.
                                  Working...
                                  X