Store must be open for this operation

Mar 16, 2011 at 9:41 PM

I have a .NET console application that uses EPPlus. Running more then one instances of the console application produces the error "Store must be open for this operation".  It ocurrs when performing a save on the package. Any ideas? The stack trace is;

at System.IO.IsolatedStorage.IsolatedStorageFile.Lock()

at System.IO.IsolatedStorage.IsolatedStorageFileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, IsolatedStorageFile isf)

at MS.Internal.IO.Packaging.PackagingUtilities.SafeIsolatedStorageFileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, ReliableIsolatedStorageFileFolder folder)

at MS.Internal.IO.Packaging.PackagingUtilities.CreateUserScopedIsolatedStorageFileStreamWithRandomName(Int32 retryCount, String& fileName)

at MS.Internal.IO.Packaging.SparseMemoryStream.EnsureIsolatedStoreStream()

at MS.Internal.IO.Packaging.SparseMemoryStream.SwitchModeIfNecessary()

at MS.Internal.IO.Packaging.SparseMemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)

at MS.Internal.IO.Packaging.CompressEmulationStream.Write(Byte[] buffer, Int32 offset, Int32 count)

at MS.Internal.IO.Packaging.CompressStream.Write(Byte[] buffer, Int32 offset, Int32 count)

at MS.Internal.IO.Zip.ProgressiveCrcCalculatingStream.Write(Byte[] buffer, Int32 offset, Int32 count)

at MS.Internal.IO.Zip.ZipIOModeEnforcingStream.Write(Byte[] buffer, Int32 offset, Int32 count)

at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)

at System.IO.StreamWriter.Write(String value)

at System.IO.TextWriter.Write(String format, Object arg0, Object arg1)

Coordinator
Mar 17, 2011 at 6:51 PM

No, I have never heard of this problem before. But it seams it is a problem with the isolated storage used by the System.Packaging API.

I can have a look at it if you create an issue and attach some code to reproduce the problem.

Jan

Mar 17, 2011 at 9:50 PM

Create a simple console application and reference the epplus library. Run two or more instance of the console application where template and newfile are commandline args (diff for each instance) and you will recieve the error. The files sizes to be produced are around 13,500KB. IsolatedStorage was also my guess but I can't find a solution. I also believe it has something to do with the file size. Running a single instance is no problem. Each instance runs under the same user/domain.

using (ExcelPackage xlPackage = new ExcelPackage(newFile, template))
{

// get the data and fill rows onwards
    foreach (DataRow drRow in dt.Rows)
     {
                               
           xlcol = 1;
           foreach (DataColumn dcCol in dt.Columns)
          {
                                   
               xlworksheet.Cell(xlrow, xlcol++).Value = drRow[dcCol];
           }
          xlrow++;
      }

xlPackage.Save();

{

 

Coordinator
Mar 18, 2011 at 2:41 PM

I had a quick search for this and it might be the isolated storage size that need to be increased.

Not sure if this will help, but check this thread...

http://forums.silverlight.net/forums/p/189926/437329.aspx

Mar 18, 2011 at 5:47 PM

I don't think it is the size. My guess is that multiple instances are sharing the same isolatedstorge file. This is because both instances running are running under the same user. So if the file is locked then I think a timeout occurs and hence the "Store must be Open". I'm running a test where I run the same EXE from two separate machines... I will let you know but if this is the case I have no clue on how to resolve it? Maybe a mutex? Or maybe create a new appdomain for each instance?

Mar 21, 2011 at 5:20 PM

I found this on the openxml website. Any ideas?

 

Hi All,

I have found a resolution to the problem I mentioned yesterday. I apologise for not being able to update.I guess I'll try to explain what was happening but would be nice if you guys can have a better explanation.

There seemed to be a problem with the' initial' Package instance, some kind of 'resource leak' has occured that .Net has reported as  Access Denied or probably something held on to a resource during the Save operation. I'll reiterate the sequence that I did to reproduce the problem at a high level.

1. Open the docx file in 'ReadWrite' mode using the Package.Open

2. Access the Document Part and Load it into a XmlDocument object

3. A helper class  builds the document.xml with the data.

4. A streamWriter is created.

5.  XMLDocument.Save(outStream) - This is where the exception was thrown on IIS.(Not on local dev server!!! Why is that ?)

Resolution

1. I modified the initial Package access mode to 'Read'

Steps 2 ,3 and 4 remain the same.

5. I closed the package instance and re-opened in 'ReadWrite' mode

6.  XmlDocument.Save(outstream) -  (End result: Success)

Basically there seemed to be a problem with the package/zip file. I don't know why though but it's narrowed down to it. Though I can get my application across to production with more testing,  I'd be keen to know what the underlying problem really is.

For those of you who have used this mechanism of Opening a package and writing to document.xml, could you try a stress test(huge data that takes a bit of time to get to the Save part) and see how it handles it.

I hope to have explained it. Again thanks for those who responded and viewed.

Cheers

Coordinator
Mar 25, 2011 at 2:56 PM

It's would be a lot of work to implement this workaround and I'm not convinced it will solve your problem.

When I get some time over I will try to do a stresstest and see if I can reproduce this error.

Mar 26, 2011 at 5:03 PM

Much appreciated. I'm pulling my hair out over this one. I have to schedule the creation of excel docs at different times or on different machines unless the isolatedstorage error occurs. Running in windows 2003 or XP with framework 3.5 doesn't allow you to increase the isolatedstorge qouta. As I said it only seems to occur with larger files, at least that is my assumption because the error doesn't occur when I run simutanously with small size files being produced so I assume I'm running into a isolatedstorage size issue. Anyway, the plans are to use this solution increasingly and I will start running into issue trying to schedule the generation of excel workbooks avoiding running the app at the same time on the same machine by the same service user. Any input from anyone reading this is so much appreciated.

Apr 23, 2012 at 1:32 PM

Dear All...

This problem is associeated the buffer server limit (IIS).

You need increase the attribute AspBufferingLimit in MetaBase.xml.

Beware when manipulate this file... you could increase the fragility at Server.

Oct 23, 2012 at 9:34 PM

fkfouri is absolutely wrong -- AspBufferingLimit is related to ASP, not ASP.NET, and has been replaced anyway with the <limits> directive.

The issue is related directly to the size of the document being produced. Above a certain size, MS.Internal.IO.Packaging.SparseMemoryStream attempts to use Isolated Storage for the zip file rather than keeping it in memory. On a development machine, where IIS generally impersonates a real user, this isn't a big problem.

But when running as Network Service or ApplicationPoolIdentity, there's something wonky with using IsolatedFileStorage. Various people have looked at registry entries, creating or changing ACLs on isolated storage temporary directories, and locking the type to prevent multi-threaded access. None of these paths seem to be working for me.

I believe the problem just gets worse when generating more than one file at a time, and in that case, I think the IsolatedFileStorage quota is the issue. If a single file is large enough, it has the same problem, and I'm not convinced yet that any file that is large enough to use IsolatedFileStorage will successfully generate over IIS in a production environment.

I'm also pulling my hair out here. I can't be sure, but I think this got worse when I switched from WOW32 to 64-bit mode in IIS -- but that may just be because I made the switch to support larger workbooks.

I think the only course of action is to re-implement the internals of Microsoft's Packaging library, but without the use on IsolatedFileStorage. The symbol libraries are available freely and would someone to create an alternative packaging API with that minor tweak, but mucking around in VS to do this is just beyond the time I can commit to learn.

Editor
Oct 23, 2012 at 9:49 PM
Edited Oct 23, 2012 at 9:58 PM

richard,

Yes, you are right.  I've done a little research too and for now I think the only way is change System.IO.Packaging to another one, like donetzip, but it isnt a simple task, there are a high coupling between epplus and packaging api. I´ve had this issue in my production server too, if you want to reproduce it more easily, you can just create a file in isolated storage, like this:


 

 

            try
            {                
                var store1 = IsolatedStorageFile.GetUserStoreForDomain();
                store1.CreateDirectory("Test1_" + DateTime.Now.Ticks.ToString());                
                IsolatedStorageFileStream isoStream1 = new IsolatedStorageFileStream(DateTime.Now.Ticks.ToString() + "_InTheRoot1.txt", FileMode.Create, store1);                
                isoStream1.Close();                
            }
            catch(Exception e)
            {   
            }
			

            try
            {
                var store2 = IsolatedStorageFile.GetUserStoreForAssembly();
                store2.CreateDirectory("Test2_" + DateTime.Now.Ticks.ToString());                
                IsolatedStorageFileStream isoStream2 = new IsolatedStorageFileStream(DateTime.Now.Ticks.ToString() + "_InTheRoot2.txt", FileMode.Create, store2);
                isoStream2.Close();                

            }
            catch (Exception e)
            {
            }


            try
            {
                var store3 = IsolatedStorageFile.GetUserStoreForApplication();
                store3.CreateDirectory("Test3_" + DateTime.Now.Ticks.ToString());
                IsolatedStorageFileStream isoStream3 = new IsolatedStorageFileStream(DateTime.Now.Ticks.ToString() + "_InTheRoot3.txt", FileMode.Create, store3);

            }
            catch (Exception e)
            {
            }			

 

 

I don't know exactly how our infra team fixed that problem, but I know they find out where create isolated storage folder and gave permissions for our app pool user.

Oct 24, 2012 at 8:10 PM

I've done a bit more research and opened up a separate Issue with more details.

http://epplus.codeplex.com/workitem/14762

(I figured a new issue would be better than continuing this one.)

As an aside, I may have found a workaround for me -- I'm now running my IIS application pool under a newly-created, local user account rather than AppPoolIdentity or Network Service. Since switching, I've been able to generate and open much larger files than before at least and haven't seen this issue again.

That's not a workable solution for some people, but it's at least an option for people generating Excel files server-side in an intranet environment.

I've suggested porting the code from Mono's implementation of System.IO.Packaging, since it is also LGPL. The developer of that code (who also contributes to EPPlus) tells me he's not sure that code is bug-free enough to use within EPPlus, but it's at least a path forward that wouldn't require a full re-implementation of this portion of the BCL.