Também me perguntei isso, mas um rápido Google descobriu isso:
Inactive memory is memory that has previously been used to run an application. OSX keeps certain things in memory just in case you launch the application again, so that it'll launch quicker. However, this inactive memory can be released to a new app as needed.