Open Question: Which MacOS apps will be broken by sandboxing?

There were some quick responses to this morning’s post about the App Store. I immediately wished I’d at least mentioned the awkward transition the Store is going through right now. Apple is demanding basic changes to the way that apps operate. It’s called “sandboxing” and it’s designed to ensure that no app, either through malice or incompetence, can interfere with any system process or anything that any other app is doing.

Which isn’t a bad goal. It’s very user-positive. But keep in mind that Mac developers have been building apps any way they like since 1984. Many apps use techniques that are perfectly harmless, but impossible to implement under sandboxing. Other apps are designed to deliver a basic, system-wide function that’s fundamentally incompatible with the concept.

I’ve been aware of this for a while and have written and spoken about it frequently. But I only just now realized that I’ve never seen a public list of apps that are broken by sandboxing. Yes, it doesn’t mean the end of the app — users can still sideload it outside of the App Store — but it does cut the app off of Apple’s “blessed and trusted” mechanism for purchasing, installing, and updating safe apps.

I will quickly categorize three standout types of “broken” apps:

1) An app that fundamentally can’t function in a sandboxed environment. Generally, system apps that do something Wonderfully Clever. It can’t work, so it won’t be updated.

2) An app that uses a Clever Trick as a shortcut. Developer will finally sit down and figure out how to make that function work within the new rules.

3) An app that’s simply old and in its “dividend” phase. Maybe it doesn’t work because of a shortcut, maybe it doesn’t work because it hasn’t been updated to completely use the modern framework. It’s not that there are any technical barriers to making it work under the new system, but at this point in the app’s life it generates enough revenue only to support simple bugfixes, and not top-to-bottom rewrites. Developer shrugs, thinks “Okay, my Austin Powers Talking Clock had a good run and now it’s time to move on” and the app dies.

I am literally minutes away from leaving my house for an unbreakable appointment (or else I’d do an exhaustive search to see if such a list already exists).

Still, I thought I’d leave an open question:

1) Do you know of an app whose future is put in jeopardy by sandboxing? (Ideally: an app that you wrote)

2) Is it so bad that there’s a real risk that the app will be frozen in its pre-sandboxed state?

Leave responses in comments.

45 thoughts on “Open Question: Which MacOS apps will be broken by sandboxing?”

  1. Sandboxing effectively broke the logging capability in Textual, an IRC client for the Mac. Logging is an essential feature, so it’s pretty frustrating. So far the developers don’t seem too keen on finding a workaround either.

  2. Yes, our application Blast (http://www.apparentsoft.com/blast) will not work in Sandbox at all.

    We’ve developed a major upgrade now, submitted to Apple for review (as a new application) and I’m not sure if Apple will allow it in the store now, since the review won’t end before June 1st. It’s not clear if the cut-off date is by initial submission of the application or by date of its review. Sigh, Apple and its secrecy.

    Overall, it’s a pretty sad situation for us developers who are affected by this, and probably for some users.

  3. I would assume that any app that records Skype audio like Audio HijackPro or WireTap Studio. Wiretap has actually been broken since Lion and they don’t appear to be able to make it in the Sandbox.

  4. The worst part of the whole situation is that I make it a habit now to ask developers what the differences are between the App Store and direct purchased versions of apps. Voodoopad, for example, loses the “Save as PDF in Voodoopad” system service due to App store regulations.

    So not only is the app store version less functional, but now I have to go to the trouble of checking with each app producer to see what the tradeoff is. Coupled with a 30% financial hit to the smaller developers I’m supporting, the extra work of researching these App Store apps hardly seems worth it.

  5. Tembo will for sure be broken by sandboxing. I hope to be able to continue providing updates after June 1. Just submitted an update yesterday.

    HoudahGeo 3.0 was initially meant for the AppStore, but was never submitted to the AppStore because there is no way to make it work with the sandbox. Just about any of HoudahGeo’s features would be limited or gone under sandboxing.

    This week I submitted 2 new applications: SnipEdges and Command-F. For both of these I am actually not sure how feasible such features would be under sandboxing. Rather than delay the project and do additional work for no benefit, I chose to work overtime to meet the deadline.

  6. There’s going to be a stampede of submissions sans sandboxing this week to get any features and fixes in before the “new app” deadline of June 1. I’m pretty sure Apple will honor the submission date if it’s in before then. But considering the stampede, I wonder how long it will take to get approval. Reviewers will be working overtime.

  7. TextSoap 7’s Universal Menu feature (it allows users to select text in any app and apply TextSoap’s cleaners directly to that text) fits into category #1. Apple has deemed the functionality the Universal Menu uses to be “fundamentally incompatible with the sandbox.” Also, as of 10.7.4, TextSoap’s ability to internally run user created Automator workflows to perform actions on text has issues with the App in the Sandbox. The Universal Menu problem is the biggest issue as the feature is among the current users’ favorite.

    Along with those two big issues, there are lots of additional headaches that fall into category #2. Here the problem is the mechanisms used in the past which were considered valid are not considered sandbox-friendly and the replacement mechanisms haven’t yet been provided by Apple. Developers will need to spend effort working around the current implementation and then do it again when Apple (finally) provides their sanctioned solution, which will likely be the only one allowed in future MAS submissions.

  8. Coda 2 was just released last week, and I’m pretty sure it would have a hard time in the sandbox because it scans, reads and writes entire folders of files rather than just the ones a user has explicitly opened using a file dialog.

    I bought it from the Mac App Store because I wanted the iCloud support, but I am concerned that some time soon there will be a bug fix release that only comes out through the Panic web site because Apple won’t approve it.

  9. Looking over the apps I use regularly the ones I suspect will have issues are: LaunchBar, Fresh, Hazel, TextExpander, BetterTouchTool, 1Password, and QuickCursor.

    I also think Panic deliberately got Coda 2 in before the cutoff to grandfather some of their features.

  10. I have yet to see anything that addresses AppleScript support. Is AppleScript dead? Can a sandboxed app call Apple Events in another application? Can you AppleScript a sandboxed app?

    Anyone?

  11. FlashFrozen by its very nature kills other running processes. No possible hope for sandboxing unless you’re happy with it simply looking pretty in the menubar.
    It’ll be stuck the way it is without changes on the App Store until Apple cleans house, or I pull it myself :(

  12. DropDMG is not sandboxable at all, as Apple’s disk image tools don’t work in the sandbox (even when used with files that don’t require special access).

    EagleFiler‘s encryption won’t work in the sandbox (due to the disk image issue). There will be limitations on what user-supplied AppleScripts can do. Indexing certain kinds of files won’t work. Once some of the outstanding sandbox bugs are fixed, I think the application will be sandboxable, albeit it will present the user with some annoying dialogs to receive access to certain files/folders. Another issue is that adopting sandboxing will require dropping support for Mac OS X 10.6, which is currently around a third of the customer base.

    Both apps already have certain limitations due to the regular App Store rules: archiving files with certain ownership/permissions, PDF services (as with VoodooPad), OpenMeta, etc.

    @Paul Turnbull 1Password has been sandboxed for quite a while now. They did have to remove some features and preferences in order to accomplish this, though.

  13. “Looking over the apps I use regularly the ones I suspect will have issues are: LaunchBar, Fresh, Hazel, TextExpander, BetterTouchTool, 1Password, and QuickCursor.”

    Most of those apps aren’t in the app store anyway so they won’t be affected.

    Also, I am pretty sure 1Password is already sandboxed, no?

  14. There’s one more type of broken app you should add to your list. Apps that rely on Apple frameworks that are broken by sandboxing. There are several Apple-supplied frameworks that do not yet fully support sandboxing. I’m sure Apple can fix these bugs in their frameworks, but they have not done so yet. Consequently, apps relying on those system frameworks can’t support sandboxing.

    My app, Pear Note (http://www.usefulfruit.com/pearnote/), is one such app. A major part of Pear Note deals with media files. In many cases, it uses the QTKit framework (Apple’s Objective-C framework for using QuickTime to record, play, and edit media files). While much of this framework works within the sandbox, the writeToFile: method, used to save edited media out to disk, does not (http://openradar.appspot.com/radar?id=1475409). Even if I tell it to write within the sandbox, it tries to do things under the hood outside the sandbox and fails. As you can imagine, the inability to save anything is a deal-breaker for an app that deals heavily with media.

  15. Some aren’t yet but given the pressure and benefits to being in the store the sandboxing issue still affects them if only by keeping them out of the store.

    TextExpander, QuickCursor, and 1Password are in the store now. 1Password is sandboxed but it did cause a some major problems for people who used Dropbox. Apple only allowed them explicit paths to save to essentially forced us to save our DropBox libraries at the root of our DropBox folders. The same issue exists with Day One.

    It reminds of the bad old days when the Documents folder became littered with tons of utility folders created by apps when they should have been using Application Support. Now instead of letting us create an Application Support in DropBox (like I used to) all those synced application folders will be cluttering up my DropBox root.

    Another app I expect to have problems if they took it to the store would be Little Snitch.

  16. We have two Mac OS X Applications on the Mac AppStore, JellyfiSSH and JellyVNC.

    Both write a file and/or use Applescript outside of it’s “Sandbox” to communicate with other applications.

    We can’t see a way around the sandboxing – but will keep the apps there and keep up the bug fixes and hopefully sneak in new features along the way!

  17. My app “Together” would lose some (more) features, mostly the more advanced stuff, the very things that were requested almost continuously until they were added.

    One in particular being its import hot key — you press a global hotkey to import something from the current app, such as a URL, email message or whatever, and that’s done using AppleScript. The same thing can be done with drag and drop or system services, but those require more work.

    The app can also store its library folder in arbitrary locations and link to files outside that folder, both totally impossible with sandboxing until 10.7.3, which made the issue far less severe, but some restrictions remain and those will doubtless confuse / irritate, along with the migration required to get the sandboxed version permission to access those files when it’s first launched.

    The biggest problem is that currently it seems apps can’t perform Spotlight searches when sandboxing is enabled. Apple has acknowledged this as a bug, but no fix has appeared yet. Together relies on Spotlight to search files’ content — a major feature — so that is my current dealbreaker for submitting a sandboxed version of the app.

    I’ve spent (and in the case of the changes that didn’t appear until 10.7.3, wasted) months sandboxing my apps. They all share a theme of working with other apps and providing a lot of flexibility in what they do. I’ve generally avoided anything too daring, without being puritanical, but it still feels the app will be restricted by sandboxing on the very features people expect and most appreciate.

  18. How about LightRoom? I have the non-Mac App Store version, so I don’t know if the App Store one is different, but LR lets you store images where ever you want. Will the new version not do that?

    Will Aperture be subjected to the same restrictions, or will it get a pass?

  19. Look it guys. It’s not the end of the world. Yes sandboxing has its downsides but it also has A LOT more upside. If sandboxing were the deathnell of computer programming, then Apple wouldn’t have a thriving App Store Economy on iOS devices. There are probably going to be some programs that won’t function properly due to the programs liberties of messing with actions that don’t make any sense i.e. An Image Editor wanting access to your address book… but a majority of applications will work fine in the new sandboxing environment and over time Apple will add additional APIs for those fringe cases where “x” program should have access to “x” function. And as stated in the article, if people want to take the liberties of allowing a program unfettered access to the whole system then there is a fully sanctioned way of doing that as well.

  20. Many of the apps mentioned are designed for more advanced users, like the people commenting. Most of these users have purchased these apps outside the MAS and will always do so bc they offer greater customizatation of the Mac OS user experience.

    I use many of the apps listed here but I don’t know anyone in my offline world that use any of these apps except 1Password, which I gifted to that person.

    While I sympathize with the trouble it is causing some developers, it’s really only trouble for them and the power users that buy their products and still will, outside the MAS. At least I hope users like will still keep buying.

    It seems that the tech blogsphere likes to hype sandboxing apps as another way to complain about Apple, its lack of “openness” and the classic walled garden meme.

    I like my “walled garden” and it keeps the junk out. I selectively let uncurated apps into my garden, the apps that have a good reputation and track record for support and updates.

    I feel the pain of the app developers that make great utilities that can’t be in the sandboxed app store, and it sucks that they may lose out on a potential growth in apps sales through the MAS. But keep up the good work and I will continue to recommend your apps.

  21. All of the apps I currently have in the Mac App Store will be affected:

    1) Remote Buddy Express (http://www.iospirit.com/products/remotebuddyexpress/)

    The Express version of Remote Buddy was originally created specifically for the Mac App Store. While the original MAS rules were already challenging for a remote control solution with its reach and depth of functionality, it’ll be impossible to sandbox. Sandboxing cuts off RBE from essential system functionality, including, but not limited to:
    a) the ability to execute any user-provided AppleScript snippets, needed for user-defined actions

    b) the ability to post keyboard and mouse events, needed to provide a virtual mouse and keyboard and control other applications with keystrokes

    c) the ability to register for mouse events (CGEventTap), needed to efficiently provide the mousespot feature for highlighting parts of presentations

    d) the ability to freely browse the file system, needed for RBE’s built-in file browser and the list of recently opened files it provides – where applicable – for all of the 100+ applications it can control

    e) the ability to use Spotlight, needed by RBE’s built-in browser of VIDEO_TS folders

    f) the ability to use the Accessibility APIs, used to control other applications. Worth noting: pretty much all tools designed to help disabled users are using the Accessibility APIs as well – and are likewise impossible to implement inside a sandbox.

    2) Spacious (http://www.iospirit.com/products/spacious)

    Spacious allows changing to other Lion fullscreen spaces with the mouse, mouse wheel and – coming up, soon – mouse gestures. Without Spacious, the user would have to use the keyboard for this task.

    Sandboxing cuts off Spacious from essential system functionality, including, but not limited to:
    a) the ability to post keyboard events to trigger the change to another fullscreen space

    b) the ability to register for and modify mouse and mouse wheel events globally (CGEventTap), needed to efficiently watch mouse movements and provide its core functionality

    It’s not possible to implement any of the listed features inside the sandbox. Going forward, I’ll try to update the Mac App Store versions alongside the versions from our website for as long as possible. I’ve also made preparations to provide updates to software purchased from the Mac App Store via our website. Finally, I’m actively looking into selling currently MAS-exclusive titles through our website. My plans for the further _development_ of Remote Buddy and Spacious aren’t changed at all by Apple’s sandbox – at least in the short to mid term. What it certainly does, however, is to add unnecessary, avoidable complexity to the development, which benefits neither Apple, nor the users, nor us developers. Really, a complete waste of time and effort that could otherwise go into the development of new, innovative features and products that benefit all of Apple, the users and us developers.

    I really hope Apple will reconsider and add sandbox entitlements for the APIs my and so many other applications completely depend upon. After all where’s the risk to allow a tool like f.ex. Spacious to send keystrokes and listen to events? I really can’t see any way to exploit anything in it, in any way.

    Best regards,
    Felix Schwarz

    P:S.: Worth noting, via the wonderful blog of MJ Tsai: not even Core Data seems to work properly (or, “safely”) in the sandbox (http://mjtsai.com/blog/2012/05/28/core-data-document-based-application-and-sandboxing/).

  22. Two of my apps, Recent Menu and Recent Redux, use custom Spotlight queries to find files and folders anywhere on local volumes that have been accessed by the user. This functionality is obviously severely affected by the Sandbox. Though there might be a way to get this working in a sandboxed environment, an implementation would be quite experimental in terms of Apple’s submission guidelines. I’ll give it a try, I think.

  23. I think that app on mac app store must be 100% safe, so if a user need something that can be dangerous, he must do a conscious choice, downloading and running it externally from the “safe way”

  24. Pretty much any app that integrates with iTunes or iPhoto is going to break. The user can put the iTunes Music Library or the iPhoto Library *anywhere*. The only way to find where it is is to read the iPhoto Preferences file (sandbox violation!).

    Even if the user happened to put these in the “standard places” and the software just looked there first, the user can choose not to copy their media into these respective libraries (like if you want your photos or music on an external hard drive you will turn off the “copy files to iTunes|iPhoto Media Library when adding to iTunes|iPhoto” setting in iTunes|iPhoto preferences. The xml interchange file used by apps integrating with iTunes & iPhoto have the paths to the actual locations of the media files. However, under sandboxing Apps cannot access these locations without prompting the user first. So even in this case the usability is likely going to be rather poor.

    Apple’s iLife & iWork apps all access this data which might be part of why iPhoto & iTunes and the apps that interact with them (iMovie, iDVD, Pages, Keynote, …) are not yet sandboxed since they will break when this happens also.

    This will impact pretty much every application/utility that wants to work with the image you have in your iPhoto (and Aperture? not sure) library or with the files in your iTunes Library.

    More than a few bugs have been entered on this topic, I’m sure. The three I filed were all marked as duplicates. I’ve also filed “sandboxing feedback” comments via the form on Apple’s web site.

    Time will tell.

  25. Addendum: The part that is annoying about this is that the vast majority of these apps/utilities only need read access to the users’s music and/or images and thus this isn’t _really_ a security risk for the most part.

    I think that Apple’s apps use the iLifeMediaBrowser private framework. So if Apple makes that work for all apps and makes it work around the above issues, this would be clean fix I think.

  26. Correct me if I’m wrong, but a lot of speculation here seems to be “this app/utility accesses files and data from other apps so it won’t be allowed.” Are you not considering the entitlements? Your sandboxed app is allowed to break the sandbox through use of entitlements. If your app declares these and is approved by Apple, then it’s ok. This is why Coda 2 can access folders full of files.

    Now I’m sure Apple hasn’t thought of every entitlement, but there seems to be a lot of speculation in these comments without considering the entitlements.

  27. @Gustav Apple has said that it will be very strict in which entitlements it allows. The ones approved for 1Password were narrower than many users would have wished. The reason Coda 2 can access everything is that it isn’t sandboxed.

  28. @Gustav – further, the “temporary” entitlements are marked just that and there is no guarantee IF and for how long Apple will approve your app for. This makes putting development resources into your app even more risky than it already was.

    And for the people who are going to reply that you can just sell it outside the app store if it isn’t approved, that equates to a divide-by-eight in sales, minimum, in our experience. Changes the financial dynamics dramatically (8x better to be in the app store).

  29. @Dave Ramsey – Re AppleScript:

    Sandboxed apps can be controlled via AppleScript, but sandboxed apps can only access specific applications (as specified in entitlements). So you could have an iTunes controller for example, but you cannot have an application like Keyboard Maestro or FastScripts or ScriptEditor that allows the user to run arbitrary AppleScripts.

    @Gustav Re File System access:

    File system access entitlements are explicitly labeled as “temporary” by Apple, which would make it highly unwise for any developer to spend time and effort on building a sandboxed application (or sandboxed version of an application) that relies on such access given any update could see Apple permanently rejecting your application.

  30. Keyboard Maestro (my application) can almost certainly never be sandboxed. I am currently waiting on Keyboard Maestro 5.3 to be approved for release, and likely that will be the last Mac App Store version of Keyboard Maestro.

    Facilities that are impossible within a sandboxed application include:

    * Typed String triggers
    * Access to arbitrary files (except via “temporary” entitlements).
    * Executing of arbitrary AppleScripts.
    * Executing of arbitrary shell scripts.
    * Probably executing Automator Workflows.
    * Probably device key triggers
    * Probably the MIDI triggers and MIDI actions.
    * Probably actions like Restart, Shutdown & Sleep and others.
    * Probably even UI actions like controlling windows or typing keystrokes.

    What’s worse, even if I could make a new version of Keyboard Maestro with some amount of this functionality removed, any current owners who upgraded would lose functionality and have no way to revert.

    What a complete mess.

  31. @Peter

    Keyboard Maestro is insanely useful, what a shame that it appears it can never be sandboxed.

    I wish they would make an exception, a permanent exception, for you guys.

  32. We presently have seven apps in the App Store: Desktop Curtain, Key Codes, Keymo, Moom, Name Mangler, Time Sink, and Usher.

    Of those seven apps, only two (Name Mangler and Desktop Curtain) may be sandboxable. The others all do things that aren’t allowed in a sandbox. Not only that, there aren’t even any entitlements to do what we need to do.

    So the future of those five apps (a few of which have been featured by Apple on the front page of the store) in the App Store is zilch: we will be able to fix bugs, but not enhance them in any way.

    We have a plan in place to migrate App Store users to the direct sales versions, because we feel those who purchased our apps are entitled to new features in addition to bug fixes.

    What grates on us most about the whole sandboxing thing is that Apple is penalizing the very people who worked hard to play by the rules: those who have their apps already in the store. We had to use only public programming interfaces, we had to follow a defined set of rules about what was allowed, and we had to survive a review by Apple employees.

    Now, suddenly, our apps are so dangerous that they must be (effectively) pulled from the store? Meanwhile, Apple’s apps aren’t sandboxed, but you can bet they’ll be allowed to make major updates.

    Rob Griffiths
    Many Tricks

  33. Similar to Many Tricks’s* excellent Moom, MercuryMover is not sandboxable. We’ll also transition users to the direct sales version. For better or worse, this sea change may push MercuryMover permanently into the “dividend” phase of its lifecycle. Having written the first version while on paternity leave with my daughter (who’s about to “graduate” from kindergarten), it’s been a pretty decent run!

    *There is only one Many Tricks.

  34. If some of these apps, such as Moom are removed from the MAS, will there be a way to redownload an old version from the MAS onto a new CPU or do we have to manually move the apps from our old CPU to our new one?

    From past experience, VLC for iOS, had to be manually transferred from CPU to CPU, after it was removed from the iTunes Ap Store.

  35. The challenges around Remote Buddy Express and Spacious (above) also apply to our application.

    Accessing arbitrary devices is just as challenging as running arbitrary scripts from a sandbox. Hopefully more of the power in the underlying enforcement mechanism can be exposed as entitlements over time.

    Simon (VMware Fusion)

  36. @Tom: We don’t plan on removing Moom from the MAS, and if for once Apple does what they said they would do, they won’t remove it either.

    So you should be able to download it from the MAS. If that ever changes, we’ll find a way for you to crossgrade to the non-MAS version. We’ve been working on that for a while (instead of working on useful things, *sigh*). And from what I’ve heard, other developers are working on similar fallback solutions.

  37. I’m bitterly disappointed with Apple.

    I can barely believe this has actually gone ahead.

    I’m really hoping for Mr Ihantko to give another update on a podcast. (The last I recall was discussion on Ewan Rankin’s show, a few weeks ago.)

    In response to http://daringfireball.net/linked/2012/06/01/rentzsch-mac-app-store

    Good piece by Wolf Rentzsch, evaluating both the pros and cons of buying Mac apps from the App Store versus direct from developers. He makes a strong case that the new sandboxing rules that went into effect today tilt things in favor of buying direct. I agree, but I’d say that’s true only for power users. For typical users, I’d argue that the sandboxing rules make the Mac App Store even more compelling (albeit at the expense of severe headaches for developers).

    .. I’m livid. I don’t consider myself to be a power user, I barely have more than one brain cell.

    I’m kind of disappointed that the Jason Snells and Andy Ihanktos haven’t expressed more ire about this; maybe they have some insight into Apple’s long term strategy.

Comments are closed.