Hi i am uploading a large file using asp.net mvc5 to server and in controler i am saving this stream into TempData.on my local machine it is working fine and successfully uploading large files.but on hosted server it fails.i have tried it with increasing MaxAllowedContent and ExcutionTimeOut Limit but none of them worked.Can anyone help me on this.
Here is my controller action
public ActionResult UploadTransactionPDF()
{
private readonly List<MemoryStream> _PdfImage = new List<MemoryStream>();
private readonly List<string> _PdfImageNames = new List<string>();
try
{
Gateway.Instance.Logger.LogInfo(string.Format(_formatProvider,"[TransactionController -> UploadTransactionPDF ]: method started : Save pdf to memory."));
if (CheckFileExist())
{
if (TempData[TransactionRef.PdfFiles.ToString()] != null)
{
_PdfImage.Clear();
_PdfImage.AddRange(TempData[TransactionRef.PdfFiles.ToString()] as List<MemoryStream>);
}
if (TempData[TransactionRef.PdfFileName.ToString()] != null && !string.IsNullOrEmpty(TempData[TransactionRef.PdfFileName.ToString()].ToString()))
{
_PdfImageNames.Clear();
_PdfImageNames.AddRange(TempData[TransactionRef.PdfFileName.ToString()] as List<string>);
}
for (int a = 0; a < Request.Files.Count; a++)
{
if (!_PdfImageNames.Contains(Request.Files[a].FileName))
{
MemoryStream ms = new MemoryStream();
Request.Files[a].InputStream.CopyTo(ms);
_PdfImage.Add(ms);
_PdfImageNames.Add(Request.Files[a].FileName);
}
}
TempData[TransactionRef.PdfFiles.ToString()] = _PdfImage;
TempData[TransactionRef.PdfFileName.ToString()] = _PdfImageNames;
ViewBag.Info = "pdf";
ViewBag.ImageUploadInfo = Language.ImageSavedMessage.ToString();
}
else
{
ViewBag.ImageUploadInfo = Language.NoFileMessage.ToString();
}
ViewBag.FileNames = _PdfImageNames;
Gateway.Instance.Logger.LogInfo(string.Format(_formatProvider, "[TransactionController -> UploadTransactionPDF ] : method exited ."));
return View("_PDFFiles");
}
catch (Exception ex)
{
string errorInfo = string.Format(_formatProvider, "[TransactionController -> UploadTransactionPDF ] : Error : '{0}' occurred while saving pdf to memory.",ex.Message);
Gateway.Instance.Logger.LogError(errorInfo,ex);
throw new Exception(errorInfo, ex);
}
}
Thanks!
can you see if the following exists in your web.config on the dev enviroment versus on the server. ddddd should be the max your request should allow. don't make it to high to where you open yourself up for DOS. the request length wouldn't just be the file size but the entire request from the client to the server.
<configuration>
<system.web>
<httpRuntime targetFramework="4.5" maxRequestLength="ddddd" />
</system.web>
</configuration>
you can localize it to the one controller action by adding a location element in between the configuration and system.web elements.
Related
In one of my Azure Web App Web API application, I am creating temp files using this code in a Get method
string path = Path.GetTempFileName();
// do some writing on this file. then read
var fileStream = File.OpenRead(path);
// then returning this stream as a HttpResponseMessage response
My question is, in a managed environment like this (not in VM), do I need to clear those temporary files by myself?
Shouldn't Azure itself supposed to clear those temp files?
Those files only get cleaned when your site is restarted.
If your site is running in Free or Shared mode, it only gets 300MB for temp files, so you could run out if you don't clean up.
If your site is in Basic or Standard mode, then there is significantly more space (around 200GB!). So you could probably get away with not cleaning up without running into the limit. Eventually, your site will get restarted (e.g. during platform upgrade), so things will get cleaned up.
See this page for some additional detail on this topic.
Maybey if you extend FileStream you can override dispose and remove it when disposed is called? That is how i'm resolving it for now. If i'm wrong let me know.
/// <summary>
/// Create a temporary file and removes its when the stream is closed.
/// </summary>
internal class TemporaryFileStream : FileStream
{
public TemporaryFileStream() : base(Path.GetTempFileName(), FileMode.Open)
{
}
protected override void Dispose(bool disposing)
{
base.Dispose(disposing);
// After the stream is closed, remove the file.
File.Delete(Name);
}
}
Following sample demonstrate how to save temp file in azure, both Path and Bolb.
Doc is here:https://code.msdn.microsoft.com/How-to-store-temp-files-in-d33bbb10
Code click here:https://github.com/Azure-Samples/storage-blob-dotnet-store-temp-files/archive/master.zip
Under part is the core logic of bolb code:
private long TotalLimitSizeOfTempFiles = 100 * 1024 * 1024;
private async Task SaveTempFile(string fileName, long contentLenght, Stream inputStream)
{
try
{
await container.CreateIfNotExistsAsync();
CloudBlockBlob tempFileBlob = container.GetBlockBlobReference(fileName);
tempFileBlob.DeleteIfExists();
await CleanStorageIfReachLimit(contentLenght);
tempFileBlob.UploadFromStream(inputStream);
}
catch (Exception ex)
{
if (ex.InnerException != null)
{
throw ex.InnerException;
}
else
{
throw ex;
}
}
}
private async Task CleanStorageIfReachLimit(long newFileLength)
{
List<CloudBlob> blobs = container.ListBlobs()
.OfType<CloudBlob>()
.OrderBy(m => m.Properties.LastModified)
.ToList();
long totalSize = blobs.Sum(m => m.Properties.Length);
long realLimetSize = TotalLimitSizeOfTempFiles - newFileLength;
foreach (CloudBlob item in blobs)
{
if (totalSize <= realLimetSize)
{
break;
}
await item.DeleteIfExistsAsync();
totalSize -= item.Properties.Length;
}
}
I our app (Xamarin C#) we download files from a server. At the end of a succeful download we get the URI to the newly-downloaded file and from the URI we get the file path:
Android.Net.Uri uri = downloadManager.GetUriForDownloadedFile(entry.Value);
path = u.EncodedPath;
In Android 4.4.2 and in Android 5 the uri and path look like this:
uri="file:///storage/emulated/0/Download/2.zip"
path = u.EncodedPath ="/storage/emulated/0/Download/2.zip"
We then use path to process the file.
The problem is that in Android 6 (on a real Nexus phone) we get a completely different uri and path:
uri="content://downloads/my_downloads/2802"
path="/my_downloads/2802"
This breaks my code by throwing a FileNotFound exception. Note that the downloaded file exists and is in the Downloads folder.
How can I use the URI I get from Android 6 to get the proper file path so I can to the file and process it?
Thank you,
donescamillo#gmail.com
I didn't get your actual requirement but it looks like you want to process file content. If so it can be done by reading the file content by using file descriptor of downloaded file. Code snippet as
ParcelFileDescriptor parcelFd = null;
try {
parcelFd = mDownloadManager.openDownloadedFile(downloadId);
FileInputStream fileInputStream = new FileInputStream(parcelFd.getFileDescriptor());
} catch (FileNotFoundException e) {
Log.w(TAG, "Error in opening file: " + e.getMessage(), e);
} finally {
if(parcelFd != null) {
try {
parcelFd.close();
} catch (IOException e) {
}
}
}
But I am also looking to move or delete that file after processing.
May you an build your URI with the download folder :
Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOWNLOADS).toURI();
It's work. #2016.6.24
#Override
public void onReceive(Context context, Intent intent) {
String action = intent.getAction();
if(DownloadManager.ACTION_DOWNLOAD_COMPLETE.equals( action)) {
DownloadManager downloadManager = (DownloadManager)context.getSystemService(Context.DOWNLOAD_SERVICE);
long downloadId = intent.getLongExtra(DownloadManager.EXTRA_DOWNLOAD_ID, 0);
DownloadManager.Query query = new DownloadManager.Query();
query.setFilterById(downloadId);
Cursor c = downloadManager.query(query);
if(c != null) {
if (c.moveToFirst()) {
int columnIndex = c.getColumnIndex(DownloadManager.COLUMN_STATUS);
if (DownloadManager.STATUS_SUCCESSFUL == c.getInt(columnIndex)) {
String downloadFileUrl = c.getString(c.getColumnIndex(DownloadManager.COLUMN_LOCAL_URI));
startInstall(context, Uri.parse(downloadFileUrl));
}
}
c.close();
}
}
}
private boolean startInstall(Context context, Uri uri) {
if(!new File( uri.getPath()).exists()) {
System.out.println( " local file has been deleted! ");
return false;
}
Intent intent = new Intent();
intent.addFlags( Intent.FLAG_ACTIVITY_NEW_TASK);
intent.setAction( Intent.ACTION_VIEW);
intent.setDataAndType( uri, "application/vnd.android.package-archive");
context.startActivity( intent);
return true;
}
i am getting below error while run this program
i am using SharePoint server 2010 and recently i am install danish language pack in SharePoint server for client environment . but after this when ever i am run below code
i am getting below exceptions
org.apache.chemistry.opencmis.commons.exceptions.CmisInvalidArgumentException: Bad Request
at org.apache.chemistry.opencmis.client.bindings.spi.atompub.AbstractAtomPubService.convertStatusCode(AbstractAtomPubService.java:453)
at org.apache.chemistry.opencmis.client.bindings.spi.atompub.AbstractAtomPubService.read(AbstractAtomPubService.java:601)
at org.apache.chemistry.opencmis.client.bindings.spi.atompub.NavigationServiceImpl.getChildren(NavigationServiceImpl.java:86)
at org.apache.chemistry.opencmis.client.runtime.FolderImpl$2.fetchPage(FolderImpl.java:285)
at org.apache.chemistry.opencmis.client.runtime.util.AbstractIterator.getCurrentPage(AbstractIterator.java:132)
at org.apache.chemistry.opencmis.client.runtime.util.AbstractIterator.getTotalNumItems(AbstractIterator.java:70)
at org.apache.chemistry.opencmis.client.runtime.util.AbstractIterable.getTotalNumItems(AbstractIterable.java:94)
at ShareTest1.main(ShareTest1.java:188)
public class ShareTest
{
static Session session = null;
static Map<String,Map<String, String>> allPropMap=new HashMap<String,Map<String, String>>();
static void getSubTypes(Tree tree)
{
ObjectType objType = (ObjectType) tree.getItem();
if(objType instanceof DocumentType)
{
System.out.println("\n\nType name "+objType.getDisplayName());
System.out.println("Type Id "+objType.getId());
ObjectType typeDoc=session.getTypeDefinition(objType.getId());
Map<String,PropertyDefinition<?>> mp=typeDoc.getPropertyDefinitions();
for(String key:mp.keySet())
{
PropertyDefinition<?> propdef=mp.get(key);
HashMap<String,String> propMap=new HashMap<String,String>();
propMap.put("id",propdef.getId());
propMap.put("displayName",propdef.getDisplayName());
System.out.println("\nId="+propMap.get("id")+" DisplayName="+propMap.get("displayName"));
System.out.println("Property Type = "+propdef.getPropertyType().toString());
System.out.println("Property Name = "+propdef.getPropertyType().name());
System.out.println("Property Local Namespace = "+propdef.getLocalNamespace());
if(propdef.getChoices()!=null)
{
System.out.println("Choices size "+propdef.getChoices().size());
}
if(propdef.getExtensions()!=null)
{
System.out.println("Extensions "+propdef.getExtensions().size());
}
allPropMap.put(propdef.getId(),propMap);
}
List lstc=tree.getChildren();
System.out.println("\nSize of list "+lstc.size());
for (int i = 0; i < lstc.size(); i++) {
getSubTypes((Tree) lstc.get(i));
}
}
}
public static void main(String[] args)
{
/**
* Get a CMIS session.
*/
String user="parag.patel";
String pwd="Admin123";
/*Repository : Abc*/
String url="http://sharepointind1:34326/sites/DanishTest/_vti_bin/cmis/rest/6B4D3830-65E5-49C9-9A02-5D67DB1FE87B?getRepositoryInfo";
String repositoryId="6B4D3830-65E5-49C9-9A02-5D67DB1FE87B";
// Default factory implementation of client runtime.
// default factory implementation
SessionFactory factory = SessionFactoryImpl.newInstance();
Map<String, String> parameter = new HashMap<String, String>();
// user credentials
parameter.put(SessionParameter.USER, "parag.patel");
parameter.put(SessionParameter.PASSWORD, "Admin123");
// connection settings
parameter.put(SessionParameter.ATOMPUB_URL, "http://sharepointind1:34326/sites/DanishTest/_vti_bin/cmis/rest/6B4D3830-65E5-49C9-9A02-5D67DB1FE87B?getRepositoryInfo");
parameter.put(SessionParameter.BINDING_TYPE, BindingType.ATOMPUB.value());
parameter.put(SessionParameter.REPOSITORY_ID, "6B4D3830-65E5-49C9-9A02-5D67DB1FE87B");
parameter.put(SessionParameter.LOCALE_ISO3166_COUNTRY, "DK");
parameter.put(SessionParameter.LOCALE_ISO639_LANGUAGE, "da");
parameter.put(SessionParameter.LOCALE_VARIANT, "");
parameter.put(SessionParameter.AUTHENTICATION_PROVIDER_CLASS, CmisBindingFactory.STANDARD_AUTHENTICATION_PROVIDER);
// create session
Session session = factory.createSession(parameter);
if(repositoryId!=null)
{
parameter.put(SessionParameter.REPOSITORY_ID, repositoryId);
session=factory.createSession(parameter);
RepositoryInfo repInfo=session.getRepositoryInfo();
System.out.println("Repository Id "+repInfo.getId());
System.out.println("Repository Name "+repInfo.getName());
System.out.println("Repository cmis version supported "+repInfo.getCmisVersionSupported());
System.out.println("Sharepoint product "+repInfo.getProductName());
System.out.println("Sharepoint version "+repInfo.getProductVersion());
System.out.println("Root folder id "+repInfo.getRootFolderId());
try
{
AclCapabilities cap=session.getRepositoryInfo().getAclCapabilities();
OperationContext operationContext = session.createOperationContext();
int maxItemsPerPage=5;
//operationContext.setMaxItemsPerPage(maxItemsPerPage);
int documentCount=0;
session.setDefaultContext(operationContext);
CmisObject object = session.getObject(new ObjectIdImpl(repInfo.getRootFolderId()));
Folder folder = (Folder) object;
System.out.println("======================= Root folder "+folder.getName());
ItemIterable<CmisObject> children = folder.getChildren();
long to=folder.getChildren().getTotalNumItems();
System.out.println("Total Children "+to);
Iterator<CmisObject> iterator = children.iterator();
while (iterator.hasNext()) {
CmisObject child = iterator.next();
System.out.println("\n\nChild Id "+child.getId());
System.out.println("Child Name "+child.getName());
if (child.getBaseTypeId().value().equals(ObjectType.FOLDER_BASETYPE_ID))
{
System.out.println("Type : Folder");
Folder ftemp=(Folder) child;
long tot=ftemp.getChildren().getTotalNumItems();
System.out.println("Total Children "+tot);
ItemIterable<CmisObject> ftempchildren = ftemp.getChildren();
Iterator<CmisObject> ftempIt = ftempchildren.iterator();
int folderDoc=0;
while (ftempIt.hasNext()) {
CmisObject subchild = ftempIt.next();
if(subchild.getBaseTypeId().value().equals(ObjectType.DOCUMENT_BASETYPE_ID))
{
System.out.println("============ SubDoc "+subchild.getName());
folderDoc++;
documentCount++;
}
}
System.out.println("Folder "+child.getName()+" No of documents="+(folderDoc));
}
else
{
System.out.println("Type : Document "+child.getName());
documentCount++;
}
}
System.out.println("\n\nTotal no of documents "+documentCount);
}
catch(CmisPermissionDeniedException pd)
{
System.out.println("Error ********** Permission Denied ***************** ");
pd.printStackTrace();
}
catch (CmisObjectNotFoundException co) {
System.out.println("Error ******** Root folder not found ***************");
co.printStackTrace();
}
catch (Exception e) {
e.printStackTrace();
}
}
else
{
System.out.println("Else");
Repository soleRepository=factory.getRepositories(
parameter).get(0);
session = soleRepository.createSession();
}
}
}
here it is my lib which i used in above code .
chemistry-opencmis-client-api-0.9.0
chemistry-opencmis-client-bindings-0.9.0
chemistry-opencmis-client-impl-0.9.0
chemistry-opencmis-commons-api-0.9.0
chemistry-opencmis-commons-impl-0.9.0
log4j-1.2.14
slf4j-api-1.6.1
slf4j-log4j12-1.6.1
it works fine when i am trying to connect repository (url) which is created in english language .
but when try to connect with the danish .repository then getting error.
The best thing you can do is to increase the SharePoint log level for CMIS. Sometimes the logs provide a clue.
The SharePoint 2010 CMIS implementaion isn't a 100% spec compliant. OpenCMIS 0.12.0 contains a few workarounds for SharePoint 2010 and 2013. Most of them a little things like an extra requied URL parameter that isn't in the spec. I wouldn't be supprised if this is something similar.
I have an application written with Xamarin.iOS that causes files to be corrupted after they are downloaded into the application.
I am comparing checksums by download the file within the application, and then uploading the downloaded file to dropbox. I am then comparing the checksum of the dropbox file to the original file on the server.
Normally, the checksums match. Occasionally, the checksums do not match. These files are really videos, and it causes the videos to not be playable.
The Question
How can downloads (using HttpWebRequest) occasionally cause corrupted files, but continue to download successfully?
The Code
public class DownloadService : IDownloadService
{
private const int BufferSize = 1024*1024;
/// <summary>
/// Download a remote file to a local location.
/// </summary>
/// <param name="url">The url of the file to download.</param>
/// <param name="localPath">The local path to save the file.</param>
/// <param name="progress">The progress off the download (percentage|bytesread|totalbytes).</param>
/// <param name="cancel">A reference to a boolean that can be set to true if you want to cancel the download.</param>
/// <returns>
/// Returns the local file that was downloaded.
/// Use this instead of the localPath parameter because you may have passed no extension asking for us to autodetermine the extension.
/// </returns>
public string DownloadFile(string url, string localPath, Action<int, long, long> progress, ref bool cancel)
{
cancel = false;
var didCancel = false;
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(url);
webRequest.ReadWriteTimeout = 3000;
var buffer = new byte[BufferSize];
var realPath = localPath;
try
{
using (var response = webRequest.GetResponse())
{
// get the total size of this request
var totalSize = 0;
try { totalSize = int.Parse(response.Headers["MediaItemLength"]);
}catch(Exception ex) { throw new NotImplementedException("Get the total size some other way!"); }
// ensure there is a proper extension
if (string.IsNullOrEmpty(Path.GetExtension(localPath)))
{
// the localpath given didn't have an extension.
// we need to try to determine the exention through custom HTTP headers.
string httpHeaderExtension = null;
try
{
httpHeaderExtension = response.Headers["MediaExtension"];
if (string.IsNullOrEmpty(httpHeaderExtension))
throw new Exception("The header was present, but it was empty.");
}
catch (Exception ex)
{
throw new Exception("localPath didn't provide an extension and we couldn't infer would through custom HTTP headers.");
}
// add the correct extension to our local file path
realPath += httpHeaderExtension;
}
if(File.Exists(realPath))
File.Delete(realPath);
using (var responseStream = response.GetResponseStream())
{
using (var dest = new FileStream(realPath, FileMode.CreateNew, FileAccess.Write))
{
long totalBytes = 0;
int currentBlockSize;
while ((currentBlockSize = responseStream.Read(buffer, 0, buffer.Length)) > 0)
{
totalBytes += currentBlockSize;
var percentage = (int)(totalBytes * 100.0 / totalSize);
dest.Write(buffer, 0, currentBlockSize);
if (progress != null)
progress(percentage, totalBytes, totalSize);
if (cancel)
{
didCancel = true;
break;
}
}
}
}
}
}catch(Exception ex)
{
// delete the file
if (File.Exists(realPath))
File.Delete(realPath);
throw;
}
// delete the file
if (didCancel && File.Exists(realPath))
File.Delete(realPath);
return realPath;
}
}
The code is pretty straight forward. I do include some additional/custom headers for getting the total file size so that I can avoid a HEAD request to get the value. I need the total file size for download progress.
I want to download the files from a sharepoint document library through code as there are thousand of files in the document library.
I am thinking of creating console application, which I will run on sharepoint server and download files. Is this approach correct or, there is some other efficient way to do this.
Any help with code will be highly appreciated.
Like SigarDave said, it's perfectly possible to achieve this without writing a single line of code. But if you really want to code the solution for this, it's something like:
static void Main(string[] args)
{
// Change to the URL of your site
using (var site = new SPSite("http://MySite"))
using (var web = site.OpenWeb())
{
var list = web.Lists["MyDocumentLibrary"]; // Get the library
foreach (SPListItem item in list.Items)
{
if (item.File != null)
{
// Concat strings to get the absolute URL
// to pass to an WebClient object.
var fileUrl = string.Format("{0}/{1}", site.Url, item.File.Url);
var result = DownloadFile(fileUrl, "C:\\FilesFromMyLibrary\\", item.File.Name);
Console.WriteLine(result ? "Downloaded \"{0}\"" : "Error on \"{0}\"", item.File.Name);
}
}
}
Console.ReadKey();
}
private static bool DownloadFile(string url, string dest, string fileName)
{
var client = new WebClient();
// Change the credentials to the user that has the necessary permissions on the
// library
client.Credentials = new NetworkCredential("Username", "Password", "Domain");
var bytes = client.DownloadData(url);
try
{
using (var file = File.Create(dest + fileName))
{
file.Write(bytes, 0, bytes.Length); // Write file to disk
return true;
}
}
catch (Exception)
{
return false;
}
}
another way without using any scripts is by opening the document library using IE then in the ribbon you can click on Open in File Explorer where you can then drag and drop the files right on your desktop!