Ghost Memories Mac OS

While reading thru some Apple forums, I found a question people asked a lot. While most responses were generally correct, I felt that all answers lacked some important details and as a result confused people even more. The purpose of this article to clarify those details as much as possible.

Mac OS X introduced concepts then foreign to Mac users, like the Dock and Terminal. As someone who lived through the transition, Mac OS X was a huge leap forward. It made not only the classic Mac OS feel dated, but also Microsoft’s competing Windows XP, which wouldn’t even ship until August of that year. Memory management in modern operating systems works similarly. In this case, a whiteboard is a physical memory which is shared by apps (teams). When apps request additional memory, and there is no more free space, the OS checks all memory pages. The OS tries to find ones that were not used recently and copies them to the disk.

First, let’s answer the first question from the title of the post.

What is wired memory?

Wired memory on Mac is reserved for the macOS kernel and its data structures, and it cannot be paged out. The normal size of the wired memory is around 2GB. User applications add on average 1MB per instance, and too many running programs can contribute to high wired memory usage.

Now, let’s dive into more technical details. Bally online slots.

When the Mac suddenly becomes too slow, the laptop owners start browsing forums and learn to check the memory tab in Activity Monitor.

The screenshot above which I made after rebooting the Mac indicates that my Mac has 8GB of Physical Memory and 2.33GB of it is in the Used Memory bucket. The used memory then consists of App Memory (1.2GB) and Wired Memory (1.13GB).

Why do people ask about Wired memory?

Wired memory is a memory that can’t be compressed or paged out to your startup drives, so it must stay in RAM. Moreover, other processes can’t borrow it.

The key phrase here is “can’t be paged out.”

Before we proceed further let me explain what paging is.

The architecture of modern computers was designed in the 20th century. Any computer has three main parts: processor, RAM (random access memory) and storage (disk, tape, punched cards, etc.).

The processor works directly with RAM because the latter is thousands of times faster than HHD or SSD. Besides the speed, there are two more differences between RAM and HHD:

  • RAM needs constant power. When a computer shuts down or restarts, the data in RAM gets wiped. Hard drives preserve data even when the computer is off.
  • RAM is much more expensive. For instance, for an additional $200, you can buy either 8GB more RAM or 128GB more SSD when purchasing a new MacBook Pro.

The last point is significant. If RAM was cheap, then Apple would stuff its laptops with tons of memory, and nobody would care to read this article. But since its more expensive than disk storage computer engineers invented memory swapping or paging.

Ghost Memories Mac OS

Let me use a whiteboard analogy.
Imagine, you work in a startup company which rents couple desks in a coworking space. Amenities in a coworking environment include unlimited coffee and beer, Wi-Fi, bike storage, and a whiteboard.

When Team A needs to brainstorm the Next Big Thing, they go to the whiteboard and start writing down some ideas. Unfortunately, there is only one whiteboard on the floor. Team B wants to brainstorm their TNBT too, and they also need some whiteboard space.

Team A graciously agrees to share some of the whiteboard space. Then another team comes, and another. With each team, the amount of free space shrinks.

When the available area gets too tight, teams decide to write down some of their ideas on the paper and clear the piece of the whiteboard for new ideas. When they need to access their old ideas, they search thru past papers and copy them to the whiteboard.

Memory management in modern operating systems works similarly. In this case, a whiteboard is a physical memory which is shared by apps (teams). When apps request additional memory, and there is no more free space, the OS checks all memory pages.

High roller casino. The OS tries to find ones that were not used recently and copies them to the disk. Apps do not write down unused memory like in whiteboard example, the OS does. When the app needs back that data, the OS reads it back to RAM from the disk.

In theory, more disk storage the computer has more RAM can be extended by paging. The reverse is also true, in case of little or no free disk space swapping is almost impossible.

Mojave

This wonderful mechanism, however, has a significant flow: disk is thousands of times slower than RAM. When there is no enough memory, the system keeps paging in and out data continuously, which causes so-called disk thrashing.

In the latter case, the computer, instead of executing programs keeps moving data making the life of a computer user miserable.

So what it has to do with wired memory? Continuing the analogy, imaging the coworking administration decided to block the quarter of the whiteboard and write a schedule of events on it. Plus they disallowed any team to wipe off the schedule.

More space the administration takes less whiteboard space teams can use. More wired memory the macOS uses fewer memory apps can share.

By the way, if you were wondering what represents Cached Files in the first screenshot, it’s a memory that was used by an app which was recently terminated. It’s as if Team A decided to go to a nearby restaurant to grab some lunch.

If by the time they come back, their ideas are still on the whiteboard, they can start from where they left. If not, then they have to copy everything from their papers back to the whiteboard. Street fighter juego gratis.

Cached Files is a mechanism which macOS uses to speed up the recently closed applications in case if a user wants to start them again. If meantime another app needs more memory, the OS will claim it from Cashed Files.

So, if you see too much Cached Files don’t worry, it’s same as free memory.

If you need proof, try following in Terminal:

  • In the Terminal app type sudo purge and hit Enter
  • Type in the admin password

When I ran this command, the only thing that has changed in Activity Monitor was Cached Files: its size dropped from 1.89GB to 594MB.

Is it ok to use sudo purge? There is no harm because all it does is clear unused memory. But it also doesn’t do anything valuable. macOS can release unused memory when apps ask for more.

What’s included in the Wired memory?

According to Apple documentation, wired memory is a part of memory reserved for OS kernel and its data structures. Following kernel objects belong to the wired memory: Virtual Memory objects, virtual memory and I/O buffer caches, various drivers.

User applications do not directly work with the hardware; they send requests to kernel drivers and those, in turn, perform hardware operations. For instance, the internet browser does not send pixels to the MacBook screen. Instead, it sends requests to the video driver, which then changes colors of pixels on the screen.

However, user apps can also influence how much of wired memory is used, although indirectly.

Following is a table with some (not all) types of objects user apps indirectly create in the wired memory. Note that the data in the table is outdated, but it at least gives some idea why the wired memory changes every time we load or unload an app.

Let me explain some of the objects in the table on the example of the Google Chrome browser. When I first started Chrome, it created several processes: one Google Chrome and 8 Google Chrome Helpers. The wired memory immediately jumped to 1.35GB.

Each line in the Activity Monitor is a process: an application instance. Each process is a collection of threads and threads perform actual work on the computer. If data in Table 1 was current, we could count that Google Chrome process added 16KB to store process data, plus 46 * 21KB = 966KB for threads.

Chrome is an interesting app. Every time you open a new tab, it starts additional helpers. Each helper is a separate process, and each process spawns numerous threads.

When I opened one tab, the number of Chrome helpers jumped from 8 to 16. After opening a second tab the number of helpers increased to 24, and after third to 26. After opening three tabs, the wired memory size increased to 1.61GB.

As you can see each process indirectly increases wired memory because macOS has to record those threads somewhere.

And this is not all. While Apple claims that wired memory is reserved for kernel and some objects which describe user processes, some applications can tap into the wired memory directly.

For instance, if you ever used virtualization software such as Parallels Desktop or VMWare, you should know that they use the wired memory directly to improve the performance.

If you think about this, it makes perfect sense. Virtual frameworks are designed to act like real machines so that OS running inside can handle the memory the way they always do. For instance, if you run macOS virtual machine on Mac, the macOS will create its version of wired memory inside VM.

After digging a bit in the Kernel Programming Guide, I found that applications can allocate memory in the kernel by calling kmem_alloc_wired function instead of regular alloc.

Now, what all of this means in practice?

How to free up memory on Mac

There are two main reasons for the high usage of wired memory:

1) Some applications modify the kernel and use wired memory deliberately. Examples are drivers and applications working on the kernel level such as antiviruses, or virtual machines. If for instance, there are some widgets in the Mission control, turn off those you don’t need.

In the case of virtual machines, the remedy is to limit the memory configuration. For example, if the Windows VM is configured to use 8GB of RAM lower it to 4GB.

2) When too many applications are active, they still use RAM. Unlike Windows, where clicking on close button unloads the app, in macOS, the app continues staying in memory. You can tell them by a little dot under the icon in the Dock.

Use Command and Q key combination to quit the app altogether. If some apps still stay in the memory rebooting the computer sometimes can fix the problem.

And finally, high wired memory usage is not always a sign of a problem. In the following example used memory was almost 31GB on a 32GB Mac, and wired memory accounted for the bulk of used (26.61GB).

However, look at the left side of the screenshot: Memory Pressure. Memory Pressure section in Activity Monitor can have one of 3 colors like a traffic light: green, yellow, and red. If memory pressure color is red, then more RAM is needed. In case if it’s green, then there is enough memory for the current load.

Wired memory does not get released immediately when an application quits. It gets cleared by a garbage collection process which may take some time, and during this time the wired memory size may be too high.

Resources:

Image credit: Flikr

(Redirected from Mac OS memory management)
'About This Computer' Mac OS 9.1 window showing the memory consumption of each open application and the system software itself.

Historically, the classic Mac OS used a form of memory management that has fallen out of favor in modern systems. Criticism of this approach was one of the key areas addressed by the change to Mac OS X.

The original problem for the engineers of the Macintosh was how to make optimum use of the 128 KB of RAM with which the machine was equipped, on Motorola 68000-based computer hardware that did not support virtual memory.[1] Since at that time the machine could only run one application program at a time, and there was no fixedsecondary storage, the engineers implemented a simple scheme which worked well with those particular constraints. That design choice did not scale well with the development of the machine, creating various difficulties for both programmers and users.

Fragmentation[edit]

The primary concern of the original engineers appears to have been fragmentation – that is, the repeated allocation and deallocation of memory through pointers leading to many small isolated areas of memory which cannot be used because they are too small, even though the total free memory may be sufficient to satisfy a particular request for memory. To solve this, Apple engineers used the concept of a relocatable handle, a reference to memory which allowed the actual data referred to be moved without invalidating the handle. Apple's scheme was simple – a handle was simply a pointer into a (non-relocatable) table of further pointers, which in turn pointed to the data.[2] If a memory request required compaction of memory, this was done and the table, called the master pointer block, was updated. The machine itself implemented two areas in memory available for this scheme – the system heap (used for the OS), and the application heap.[3] As long as only one application at a time was run, the system worked well. Since the entire application heap was dissolved when the application quit, fragmentation was minimized.

The memory management system had weaknesses; the system heap was not protected from errant applications, as would have been possible if the system architecture had supported memory protection, and this was frequently the cause of system problems and crashes.[4] In addition, the handle-based approach also opened up a source of programming errors, where pointers to data within such relocatable blocks could not be guaranteed to remain valid across calls that might cause memory to move. This was a real problem for almost every system API that existed. Because of the transparency of system-owned data structures at the time, the APIs could do little to solve this. Thus the onus was on the programmer not to create such pointers, or at least manage them very carefully by dereferencing all handles after every such API call. Since many programmers were not generally familiar with this approach, early Mac programs suffered frequently from faults arising from this.[5]

Palm OS and 16-bit Windows use a similar scheme for memory management, but the Palm and Windows versions make programmer error more difficult. For instance, in Mac OS, to convert a handle to a pointer, a program just de-references the handle directly, but if the handle is not locked, the pointer can become invalid quickly. Calls to lock and unlock handles are not balanced; ten calls to HLock are undone by a single call to HUnlock.[6] In Palm OS and Windows, handles are an opaque type and must be de-referenced with MemHandleLock on Palm OS or Global/LocalLock on Windows. When a Palm or Windows application is finished with a handle, it calls MemHandleUnlock or Global/LocalUnlock. Palm OS and Windows keep a lock count for blocks; after three calls to MemHandleLock, a block will only become unlocked after three calls to MemHandleUnlock.

Addressing the problem of nested locks and unlocks can be straightforward (although tedious) by employing various methods, but these intrude upon the readability of the associated code block and require awareness and discipline on the part of the coder.

Memory leaks and stale references[edit]

Awareness and discipline are also necessary to avoid memory 'leaks' (failure to deallocate within the scope of the allocation) and to avoid references to stale handles after release (which usually resulted in a hard crash—annoying on a single-tasking system, potentially disastrous if other programs are running).

Switcher[edit]

The situation worsened with the advent of Switcher, which was a way for a Mac with 512KB or more of memory to run multiple applications at once.[7] This was a necessary step forward for users, who found the one-app-at-a-time approach very limiting. Because Apple was now committed to its memory management model, as well as compatibility with existing applications, it was forced to adopt a scheme where each application was allocated its own heap from the available RAM.[8]The amount of actual RAM allocated to each heap was set by a value coded into the metadata of each application, set by the programmer. Sometimes this value wasn't enough for particular kinds of work, so the value setting had to be exposed to the user to allow them to tweak the heap size to suit their own requirements. While popular among 'power users', this exposure of a technical implementation detail was against the grain of the Mac user philosophy. Apart from exposing users to esoteric technicalities, it was inefficient, since an application would be made to grab all of its allotted RAM, even if it left most of it subsequently unused. Another application might be memory starved, but would be unable to utilize the free memory 'owned' by another application.[3]

While an application could not beneficially utilize a sister application's heap, it could certainly destroy it, typically by inadvertently writing to a nonsense address. An application accidentally treating a fragment of text or image, or an unassigned location as a pointer could easily overwrite the code or data of other applications or even the OS, leaving 'lurkers' even after the program was exited. Install google file stream. Such problems could be extremely difficult to analyze and correct.

Switcher evolved into MultiFinder in System 4.2, which became the Process Manager in System 7, and by then the scheme was long entrenched. Apple made some attempts to work around the obvious limitations – temporary memory was one, where an application could 'borrow' free RAM that lay outside of its heap for short periods, but this was unpopular with programmers so it largely failed to solve the problems. Apple's System 7 Tune-up addon added a 'minimum' memory size and a 'preferred' size—if the preferred amount of memory was not available, the program could launch in the minimum space, possibly with reduced functionality. This was incorporated into the standard OS starting with System 7.1, but still did not address the root problem.[9]

Virtual memory schemes, which made more memory available by paging unused portions of memory to disk, were made available by third-party utilities like Connectix Virtual, and then by Apple in System 7. This increased Macintosh memory capacity at a performance cost, but did not add protected memory or prevent the memory manager's heap compaction that would invalidate some pointers.

32-bit clean[edit]

Originally the Macintosh had 128 kB of RAM, with a limit of 512 kB. This was increased to 4 MB upon the introduction of the Macintosh Plus. These Macintosh computers used the 68000 CPU, a 32-bit processor, but only had 24 physical address lines. The 24 lines allowed the processor to address up to 16 MB of memory (224 bytes), which was seen as a sufficient amount at the time. The RAM limit in the Macintosh design was 4 MB of RAM and 4 MB of ROM, because of the structure of the memory map.[10] This was fixed by changing the memory map with the Macintosh II and the Macintosh Portable, allowing up to 8 MB of RAM.

Because memory was a scarce resource, the authors of the Mac OS decided to take advantage of the unused byte in each address. The original Memory Manager (up until the advent of System 7) placed flags in the high 8 bits of each 32-bit pointer and handle. Each address contained flags such as 'locked', 'purgeable', or 'resource', which were stored in the master pointer table. When used as an actual address, these flags were masked off and ignored by the CPU.[4]

While a good use of very limited RAM space, this design caused problems when Apple introduced the Macintosh II, which used the 32-bit Motorola 68020 CPU. The 68020 had 32 physical address lines which could address up to 4 GB (232 bytes) of memory. The flags that the Memory Manager stored in the high byte of each pointer and handle were significant now, and could lead to addressing errors.

In theory, the architects of the Macintosh system software were free to change the 'flags in the high byte' scheme to avoid this problem, and they did. For example, on the Macintosh IIci and later machines, HLock() and other APIs were rewritten to implement handle locking in a way other than flagging the high bits of handles. But many Macintosh application programmers and a great deal of the Macintosh system software code itself accessed the flags directly rather than using the APIs, such as HLock(), which had been provided to manipulate them. By doing this they rendered their applications incompatible with true 32-bit addressing, and this became known as not being '32-bit clean'.

In order to stop continual system crashes caused by this issue, System 6 and earlier running on a 68020 or a 68030 would force the machine into 24-bit mode, and would only recognize and address the first 8 megabytes of RAM, an obvious flaw in machines whose hardware was wired to accept up to 128 MB RAM – and whose product literature advertised this capability. With System 7, the Mac system software was finally made 32-bit clean, but there were still the problem of dirty ROMs. The problem was that the decision to use 24-bit or 32-bit addressing has to be made very early in the boot process, when the ROM routines initialized the Memory Manager to set up a basic Mac environment where NuBus ROMs and disk drivers are loaded and executed. Older ROMs did not have any 32-bit Memory Manager support and so was not possible to boot into 32-bit mode. Surprisingly, the first solution to this flaw was published by software utility company Connectix, whose 1991 product MODE32 reinitialized the Memory Manager and repeated early parts of the Mac boot process, allowing the system to boot into 32-bit mode and enabling the use of all the RAM in the machine. Apple licensed the software from Connectix later in 1991 and distributed it for free. The Macintosh IIci and later Motorola based Macintosh computers had 32-bit clean ROMs.

It was quite a while before applications were updated to remove all 24-bit dependencies, and System 7 provided a way to switch back to 24-bit mode if application incompatibilities were found.[3] By the time of migration to the PowerPC and System 7.1.2, 32-bit cleanliness was mandatory for creating native applications and even later Motorola 68040 based Macs could not support 24-bit mode.[6][11]

Object orientation[edit]

The rise of object-oriented languages for programming the Mac – first Object Pascal, then later C++ – also caused problems for the memory model adopted. At first, it would seem natural that objects would be implemented via handles, to gain the advantage of being relocatable. These languages, as they were originally designed, used pointers for objects, which would lead to fragmentation issues. A solution, implemented by the THINK (later Symantec) compilers, was to use Handles internally for objects, but use a pointer syntax to access them. This seemed a good idea at first, but soon deep problems emerged, since programmers could not tell whether they were dealing with a relocatable or fixed block, and so had no way to know whether to take on the task of locking objects or not. Needless to say this led to huge numbers of bugs and problems with these early object implementations. Later compilers did not attempt to do this, but used real pointers, often implementing their own memory allocation schemes to work around the Mac OS memory model.

While the Mac OS memory model, with all its inherent problems, remained this way right through to Mac OS 9, due to severe application compatibility constraints, the increasing availability of cheap RAM meant that by and large most users could upgrade their way out of a corner. The memory was not used efficiently, but it was abundant enough that the issue never became critical. This is ironic given that the purpose of the original design was to maximise the use of very limited amounts of memory. Mac OS X finally did away with the whole scheme, implementing a modern sparse virtual memory scheme. A subset of the older memory model APIs still exists for compatibility as part of Carbon, but maps to the modern memory manager (a thread-safe malloc implementation) underneath.[6] Apple recommends that Mac OS X code use malloc and free 'almost exclusively'.[12]

References[edit]

  1. ^Hertzfeld, Andy (September 1983), The Original Macintosh: We're Not Hackers!, retrieved May 10, 2010CS1 maint: discouraged parameter (link)
  2. ^Hertzfeld, Andy (January 1982), The Original Macintosh: Hungarian, archived from the original on June 19, 2010, retrieved May 10, 2010CS1 maint: discouraged parameter (link)
  3. ^ abcmemorymanagement.org (December 15, 2000), Memory management in Mac OS, archived from the original on May 16, 2010, retrieved May 10, 2010CS1 maint: discouraged parameter (link)
  4. ^ abHertzfeld, Andy, The Original Macintosh: Mea Culpa, retrieved May 10, 2010CS1 maint: discouraged parameter (link)
  5. ^Apple Computer (October 1, 1985), Technical Note OV09: Debugging With PurgeMem and CompactMem, retrieved May 10, 2010CS1 maint: discouraged parameter (link)
  6. ^ abcLegacy Memory Manager Reference, Apple Inc., June 27, 2007, retrieved May 10, 2010CS1 maint: discouraged parameter (link)
  7. ^Hertzfeld, Andy (October 1984), The Original Macintosh: Switcher, retrieved May 10, 2010CS1 maint: discouraged parameter (link)
  8. ^Mindfire Solutions (March 6, 2002), Memory Management in Mac OS(PDF), p. 2, retrieved May 10, 2010CS1 maint: discouraged parameter (link)
  9. ^'System 7.1 upgrade guide'(PDF). Archived from the original(PDF) on March 4, 2016. Retrieved May 26, 2015.
  10. ^'memory maps'. Osdata.com. March 28, 2001. Retrieved May 11, 2010.CS1 maint: discouraged parameter (link)
  11. ^Apple Computer (January 1, 1991), Technical Note ME13: Memory Manager Compatibility, retrieved May 10, 2010CS1 maint: discouraged parameter (link)
  12. ^Memory Allocation Recommendations on OS X, Apple Inc, July 12, 2005, retrieved September 22, 2009CS1 maint: discouraged parameter (link)

External links[edit]

Ghost Memories Mac Os Download

  • Macintosh: ROM Size for Various Models, Apple Inc, August 23, 2000, retrieved September 22, 2009CS1 maint: discouraged parameter (link)

Ghost Memories Mac Os Download

Retrieved from 'https://en.wikipedia.org/w/index.php?title=Classic_Mac_OS_memory_management&oldid=1008965847'