How to upload or download very large files with out any issue ?

How to upload or download very large files with out any issue ?

Things to consider for uploading and downloading the large files to UCM

Many configuration required for dealing with very large file ( more than 10gb) . It also involves many factors like network speed , UCM server capacity , weblogic configuration etc . This blog to help get more data and set the configuration

1. Make server Ready

12C server is ready by default . No need to install any patches on weblogic server and on Oracle_ECM

For 11g server , Below patches should be installed

For upload : 16083651 & 16960063

For Download : 14339868 ( this is only for Native File download , web layout works with out patch)

2. Setting required in the client

In 64 bit machine

Use Chrome 64 bit browser , that is best option for uploading file larger than 4 GB

IE 11 , Latest FireFox might work

In 32 bit machine

It allows only 2 GB size for upload . So we have to enable Chunking for upload. It is basically split the files based on the value set . Server will club all the parts together . It is more time consuming than browser upload .

It is based on Java applet . So browser and your network should support applet usage

To enable Chunking , add below entries in config.cfg of UCM

AppletChunkThreshold=10000000 // size in bytes , chucking will start only after this size

AppletChunkSize=10000000 // each size of the packets send from client

ChunkedRequestTrace=true

MultiUpload=true

DisableHttpUploadChunking=false

3. Setting Max time for weblogic thread

We have to increase the thread time out in weblogic console . By default value is 10 min ( 600 sec), if the upload or download take more than 10 min , weblogic will kill the thread ,upload process will stopped .

How to set : Weblogic console → Servers → UCM server → Configration → Tunning

Set : Stuck Thread max time

4. How to get Value for “Stuck Thread max time “

This is specific to each environment . Get the maximum file to upload or download

Test this RIDC upload code .

Get the RIDC code mention here :

http://wccandlinux.blogspot.in/2016/04/ridc-sample-code-toget-network-speed.html

it will show the network speed and Server Capacity , So “ Stuck Thread max time” should be more than total time required for upload . In the example

Time took in UCM server is 45.422 sec

Processing Capacity of your environment 34459.44696402624 KB/s

Processing Capacity of your environment 34.45944696402624 MB/s

So here no need to change the value . If it more than 600 sec , then “ Stuck Thread max time” should be edited .

5. Get the optimal value for AppletChunkThreshold & AppletChunkSize

We have to do trail and error method . Providing details from different test cases

Size of the file to upload to UCM : 13 gb

Upload Method : Using Chunk Applet

Test Case 1 :
AppletChunkThreshold=10000000
AppletChunkSize=10000000  ( 10 MB)
StuckThreadMaxTime=600 sec ( 10 min) (default value )
Even thoug it shows Checkin is fine but no time spend in the logs . Which means check in is failed . And it failed due to weblogic thread time out .
Test Case 2.a :
AppletChunkThreshold=10000000
AppletChunkSize=1000000000  ( 1GB)
Upload failed in the client system itself

Test Case 2.b :
AppletChunkThreshold=10000000
AppletChunkSize=500000000  ( 500 MB)
Upload cause performance issues in in client . So I stopped the upload

Test Case 3 :
AppletChunkThreshold=10000000
AppletChunkSize=100000000  ( 100 MB)
This causing performance issues in server . Server CPU went up to 100%

Test Case 4 :
AppletChunkThreshold=10000000
AppletChunkSize=50000000  ( 50 MB)
StuckThreadMaxTime=900 sec ( 15 min)
Checkin failed in the system due to thread time out in weblogic .
Last CHUNKED_UPLOAD finished it . It took 528 sec
CHUNKED_UPLOAD [dUser=weblogic][IsJava=1] 528.179931640625(secs)
Test Case 5 :

AppletChunkThreshold=1000000000
AppletChunkSize=40000000 ( 40 MB)
StuckThreadMaxTime=1200 sec ( 20 min)
I am able to check in the file in the UCM with this settings .
Total time : 1070.6832275390625(secs)
Last Chunk : 526.7379150390625(secs)
>requestaudit/6 03.11 11:54:51.750 IdcServer-10 CHECKIN_NEW [dID=1054][dDocName=STJACOBPC1IDCO001218][dDocTitle=WSS][dUser=weblogic][dSecurityGroup=Public][StatusCode=0][StatusMessage=Successfully checked in content item 'STJACOBPC1IDCO001218'.][IsJava=1] 1070.6832275390625(secs)

>requestaudit/6 03.11 11:54:51.763 IdcServer-349 CHUNKED_UPLOAD [dUser=weblogic][IsJava=1] 526.7379150390625(secs)

Last Chunk is the process which will club all the parts together , it is really time consuming process .This can be saved if 64 bit chrome browser is used