Existing users, log in.  New users, create a free account.  Lost password?


MacFixIt Logo
 
Contact Us | About MacFixIt | Who's Online  

Page 1 of 2 1 2 >
Topic Options
#443595 - 06/17/08 03:06 PM jnk ::: identify or erase useless nibs and plists
Hal Itosis Offline
MacWizard

Registered: 08/23/99
Posts: 7032
Loc: 10.5.7 (build 9J61)
Did you know that the 10.5.2 update contained 120 MegaBytes of utterly useless junk?
Gobs of garbage *nobody's* Mac will ever use. Think about that for a second: 120MB

My measurement of the 10.5.3 update showed some 2421 items, wasting 154.15 megabytes
of disk space. (It may have been even more, but many languages were already cleared out of
my /System beforehand). Anyway... I deleted it the same day I updated, so it's all gone now.

Many apps are similarly over-bloated. (iChat, iWeb, Mail, etc., as well as 3rd-party stuff).

But... this isn't about language localizations (although each language contains its own junk).
This is about *development* code and resources... stuff that should never "ship".

Articles like these:
Slop -Aug 2004
Top Ten Cleanups -May 2005
Every Bundle Counts! -Mar 2007
Building and Packaging Native OS X Applications -May 2007
Why Cocoa Bloat is Nonsense -Nov 2007
Cocoa Bloat: How You Get Rid of It -Nov 2007
designable.nib -Dec 2007
A Little Goes a Long Way -Apr 2008

...inspired apps like these:
Xslimmer
Trimmit!

...but, I wanted my own command-line tool.

So here it is.

jnk does not do all the "optimizations" some other programs offer:
  • It does NOT trim or even touch binary architectures (ppc, i386, etc.)
  • It does NOT reduce any .tiff files.
  • It also leaves the PkgInfo files in place. (I discovered some programs still
    rely on the existence of PkgInfo, even though the same data is in Info.plist).
These are the items jnk will list, measure and -- when you ask it to -- erase:
  • ._*
  • .svn
  • .DS_Store
  • info.nib
  • classes.nib
  • designable.nib
  • data.dependency
  • pbdevelopment.plist
  • Resources Disabled
  • (framework) Headers

For example, mounting the Cocktail 4.1.1 dmg, and taking its measurements:

$ jnk /Volumes/Cocktail/Cocktail.app
got junk: /Volumes/Cocktail/Cocktail.app

number of info.nib items found: 3
12K total

number of classes.nib items found: 3
12K total

number of designable.nib items found: 19
4336K total

number of data.dependency items found: 6
24K total

number of Headers items found: 2
68K total

found: 33 items, wasting 4.35 megabytes of disk space.
a difference of 46.6% compared to the original 9.32MB


Imagine that... almost 50% of its 9 megabytes is a total waste!!!
[some have expressed similar sentiments about the other 50%.]
Note that in Cocktail's case, almost all the waste was in designable.nibs.
And this already is a mono-lingual package (it's only English there folks).

In general -- to clear out junk -- use a language removal tool first. Then
run this script, to see what's still left. You will get surprised sometimes.
[As has been discussed in threads here, certain apps don't like it when
we remove some languages... e.g., FontExplorer wants its German.]

Most apps don't mind if you reduce their clutter. The only one I've seen complain
so far is Pacifist (2.5.2). Unfortunately, Sir Charles has code-signed his app with
some useless nib files still inside (so you can keep them safe for him, i guess?).
Because... the developer is the *only* one who can use these files.

ohwell.

I've tested this one pretty extensively.
Use it on small items if you like... one app at a time.
You have to type -E before it will try erasing anything.
Otherwise, it just takes measurements.
[run jnk -h for usage info]
See what you find...

 Code:
#!/bin/bash -f -
# jnk  :::  identify or erase useless nib files and plists, etc.
#(c)EF/-HI-2008.Apr.15
IFS=$' \t\n'
PATH=/bin:/usr/bin
export PATH
OWD=$(pwd -P)
PROG=$(basename "$0")
XCOD=0

exit2sh ()
{
	trap '' INT QUIT ABRT TERM HUP
	[[ $1 ]] && (( XCOD+=$1 ))
	if [ $XCOD -ne 0 ]
	then
		s=s; [ $XCOD -eq 1 ] && s=
		printf '\n\e[37;41m %d error%s occurred \e[0m\n' $XCOD "$s" >&2
		[ $XCOD -eq 1 ] && XCOD=2 # exit 1 reserved to mean "got junk"
		[ $XCOD -gt 125 ] && XCOD=125
	fi
	exit $XCOD
}

trap 'printf "\r$PROG: stopping on signal\n" >&2; exit2sh' INT QUIT ABRT TERM
trap 'printf "\e[0m" >&2; stty echo -igncr' EXIT
stty -echo igncr

o=o; [ "$(ls -do)" != '.' ] && o=O
CMMND=/bin/ls\ -AeFGhkl${o}
LNGLS=
ERASE=
FOUND=
USAGE=0
H=Headers
SLIST='	._*
	.svn
	.DS_Store
	info.nib
	classes.nib
	designable.nib
	data.dependency
	pbdevelopment.plist
	Resources Disabled
'

QuitHelp ()
{
	printf '\n\e[7m about: %s \e[0m\n\n' "$PROG" >&2
	printf 'Bash script to list or erase space-wasting nib files, p' >&2
	printf 'lists,\nand misc items *inside* Mac OS X bundles.  More' >&2
	printf ' specifically:\n\n%s\t(framework) %s\n' "$SLIST" $H >&2
	printf '\nUsage:\t\e[1m%s \e[0m [\e[1m-l\e[0m | ' "$PROG" >&2
	printf '\e[1m-E\e[0m]  [\e[4m/folder/path\e[0m  \e[4m...\e[0m]\n' >&2
	printf '\nWith no option, output summarizes the total count and ' >&2
	printf 'size of\nfound items. \e[1m-l\e[0m does a long ls -l ' >&2
	printf 'listing. \e[1m-E\e[0m erases the junk.\n\nexit codes:' >&2
	printf '\n    0       = CLEAN: no junk... or, junk erased without' >&2
	printf ' error.\n    1       = DIRTY: found junk, but *no* errors' >&2
	printf '.\n    2+      = ERROR: at least one error occurred.\n\n' >&2
	exit 2
}

PostErr ()
{
	(( XCOD+=1 ))
	printf '%s:\e[37;41m ERROR \e[0m %s\n' "$PROG" "$1" >&2
}

FlashMess ()
{
	local txt='please wait...'
	[[ $1 ]] && txt=$1
	printf '\e[5m %s \e[1m%s\e[0m\r' "$txt" "$2" >&2
}

ClearMess () { printf '\e[2K\e[0m' >&2; }

ScaleNum ()
{
	echo $1 $2 |
	awk '{	z=length($2)
		if ($1>1048575) { printf("%.3g", $1/1048576)
			if (z>0) print "GB"; else print " gigabytes"
		}
		else if ($1>1023) { printf("%.3g", $1/1024)
			if (z>0) print "MB"; else print " megabytes"
		}
		else { printf("%d", $1)
			if (z>0) print "KB"; else print " kilobytes"
		}
	}'
}

new2null () { echo "$1" |tr '\n' '\000'; }

FindJunk ()
{
	local -i ctr=0 sub=0 tot=0
	FlashMess
	tmp=$(du -sk); (( XCOD+=$? ))
	ClearMess
	local siz=$(echo $tmp |awk '{ print $1 }') j
	IFS=$'\n'
	for j in $SLIST $H
	do
		IFS=$' \t\n'
		local -i c=0
		j=$(echo "$j" |sed 's:^'$'\t''*::')
		FlashMess "looking for" "$j"
		case $j in
		  $H)	FOUND=$(find -xE . -regex '.*/Contents/.*\.framework/.*' \
				-regex ".*/(Private)?$j")
			(( XCOD+=$? ))
			;;
		   *)	FOUND=$(find -xE . -regex '.*(/Contents|\.framework)/.*' \
				-name "$j")
			(( XCOD+=$? ))
			;;
		esac
		ClearMess
		[[ $FOUND ]] && c=$(new2null "$FOUND" |xargs -0 ls -1d |wc -l) &&
			[ $ctr -eq 0 ] && printf 'got junk: \e[4m%s\e[0m\n\n' "$PWD"
		[ $c -gt 0 ] &&
			printf 'number of \e[1m%s\e[0m items found: %d\n' "$j" $c
		(( ctr+=$c ))
		if [[ $FOUND ]]
		then
			sub=$(new2null "$FOUND" |xargs -0 du -ck |sed '$!d;s:[^0-9]::g')
			(( XCOD+=$? ))
			printf '%dK total\n' $sub
			(( tot+=$sub ))
			if [[ $LNGLS || $ERASE ]]
			then
				if [[ $ERASE ]]
				then
					printf 'ERASING all %s items in %s...\n' "$j" "$PWD"
				else
					[ $c -eq 1 ] && [[ $j = 'Resources Disabled' ||
					$j = $H || $j = .svn ]] && printf '%s:\n' "$FOUND"
				fi
				new2null "$FOUND" |xargs -0 $CMMND
				(( XCOD+=$? ))
			fi
		fi
		[ $c -ne 0 ] && echo
	done
	if [ $ctr -ne 0 ]
	then
		(( USAGE+=$tot ))
		local s=s; [ $ctr -eq 1 ] && s=
		local t=found; [[ $ERASE ]] && t=ERASED
		printf '\e[7m%s: %d item%s, wasting %s of disk space.\e[0m\n' \
			$t $ctr "$s" "`ScaleNum $tot`"
		echo "$siz $tot" |
		awk '{	printf "a difference of %.1f%% ", 100*$2/$1
			printf "compared to the original " }'
		ScaleNum $siz -HI-
	fi
}

case $1 in
	-l)	LNGLS=true
		shift
		;;
	-E)	ERASE=true
		CMMND=/bin/rm\ -fR
		shift
		;;
	-[hH]*)	QuitHelp
		;;
	-*)	PostErr "invalid option '${1}'"
		sleep 2
		QuitHelp
		;;
esac

k=0
if [ $# -eq 0 ]
then
	FindJunk
	(( XCOD+=$? ))
else
	while [ $# -gt 0 ]
	do
		if [[ -d $1 ]]
		then
			if cd "$1" 2>/dev/null
			then
				(( k+=1 ))
				FindJunk
				(( XCOD+=$? ))
				cd "$OWD"
			else
				PostErr "directory '$1' <--access DENIED!"
			fi
		else
			PostErr "directory? '$1' <--can't get there from here."
		fi
		shift
		[ $USAGE -ne 0 ] && [ $# -gt 0 ] && printf '__\n\n'
	done
fi

[ $XCOD -ne 0 ] || [ $USAGE -eq 0 ] || [[ $ERASE ]] && exit2sh

[ $k -gt 1 ] &&
	printf '\n\e[4mdisk waste sum for all %d arguments was %s\e[0m\n' \
		$k "`ScaleNum $USAGE`"
exit 1




Top
#443599 - 06/17/08 04:20 PM Re: jnk ::: identify or erase useless nibs and pli [Re: Hal Itosis]
Hal Itosis Offline
MacWizard

Registered: 08/23/99
Posts: 7032
Loc: 10.5.7 (build 9J61)
 Quote:
found: 33 items, wasting 4.35 megabytes of disk space.
a difference of 46.6% compared to the original 9.32MB
Imagine that... almost 50% of its 9 megabytes is a total waste!!!


That "50%" figure is in fact generous, and doesn't fully reflect the situation.
46.6% actually represents the "savings" we obtain, compared to the original.

Consider for a moment that -- without junk -- Cocktail 4.1.1 now measures 5 megs.

A more critical (fair!!!) way of looking at this is:

That same 4 megs which we "saved" (by erasing it) also represents an *80%* excess bloat,
when compared to the 5 meg item we end users should have received in the first place!!!

-HI-

Top
#443740 - 06/19/08 06:30 PM Re: jnk ::: identify or erase useless nibs and pli [Re: Hal Itosis]
CharlesS Offline
MacReporter

Registered: 07/25/00
Posts: 368
You do know that if you do this on Leopard, you'll break the code signing for every app you run it on, right?


Edited by CharlesS (06/19/08 07:09 PM)

Top
#443751 - 06/19/08 08:22 PM Re: jnk ::: identify or erase useless nibs and pli [Re: CharlesS]
Hal Itosis Offline
MacWizard

Registered: 08/23/99
Posts: 7032
Loc: 10.5.7 (build 9J61)
Yup.
SO?

--

As far as the firewall goes, a reauthorization may be needed here and there.

As mentioned, a few apps may be over-protecting themselves... and it would
be nice if they either wouldn't do that, or at least ship clean. Apple's own stuff
isn't that paranoid (every app on my Mac -- including Installer.app -- has been
slimmed, and all works beautifully). Other than having to reinstall Pacifist...
there's been no problem.



Edited by Hal Itosis (06/19/08 09:00 PM)

Top
#443754 - 06/19/08 09:11 PM Re: jnk ::: identify or erase useless nibs and pli [Re: Hal Itosis]
CharlesS Offline
MacReporter

Registered: 07/25/00
Posts: 368
It's not just the firewall - you'll also cause more subtle problems. The Keychain, for instance, relies on code-signing to remember whether an app has permission to access it or not. You're going to have weird little things happen, and you'll have a worse computing experience overall, all to save a tiny amount of hard disk space.

Code signing exists to prevent security problems that arise from unauthorized parties tampering with programs. Anything that needs to be trusted in order to work properly is going to degrade to some extent if you break the code-signing. In the future, I expect Apple to switch something on so that apps with invalid code-signing refuse to run. It's the next logical step, and something I've got a head-start on with Pacifist.

And in case you're wondering, no, I'm not going to stop checking Pacifist's code-signing on startup. Pacifist is an app that asks for the user's password and gets root access. It is something that I do not want hackers or viruses hijacking. If someone were to do so, it could cause some very bad things. I'm also not going to stop shipping the .nib files in editable format, because my localizers appreciate being able to open them, and because the extra files only total about 600-700 KB anyway, which is trivial on today's storage devices. That amount would fit on a double-density floppy disk, for crying out loud.

Top
#443756 - 06/19/08 09:58 PM Re: jnk ::: identify or erase useless nibs and pli [Re: CharlesS]
Hal Itosis Offline
MacWizard

Registered: 08/23/99
Posts: 7032
Loc: 10.5.7 (build 9J61)
 Originally Posted By: CharlesS
The Keychain, for instance, relies on code-signing to remember whether an app has permission to access it or not. You're going to have weird little things happen, and you'll have a worse computing experience overall,

That's more spin than substance.

First of all, I didn't invent this idea. Apps (linked to above) have been around since Leopard debuted,
and yes, the keychain may also get confused (if it's being used). So... we just reauthorize (or restart)
and get on with it. What else? Oh yeah, backups will see the apps as being changed, and re-backup.
All worth it, IMHO. [i'm not forcing anyone to do this... but yes, there's no free lunch here.]



 Originally Posted By: CharlesS
I expect Apple to switch something on so that apps with invalid code-signing refuse to run. It's the next logical step, and something I've got a head-start on with Pacifist.

Fair enough.

And the more aware users are of the excess excrement being dumped on their HD, the more
motivated Apple and developers will be to clean up their act. At least I certainly hope so.



 Originally Posted By: CharlesS
And in case you're wondering, no, I'm not going to stop checking Pacifist's code-signing on startup. Pacifist is an app that asks for the user's password and gets root access. It is something that I do not want hackers or viruses hijacking. If someone were to do so, it could cause some very bad things.

That's fine.



 Originally Posted By: CharlesS
I'm also not going to stop shipping the .nib files in editable format, because my localizers appreciate being able to open them,

Well, that's your willful decision... not something without any alternatives.
There are many other methods you could use, to provide that info to them.



 Originally Posted By: CharlesS
and because the extra files only total about 600-700 KB anyway, which is trivial on today's storage devices. That amount would fit on a double-density floppy disk, for crying out loud.

Floppy disk? \:\/

The useless nib files in the last 2 Leopard updates were probably bigger than
System 7, System 8 and Mac OS 9 put together. [120MB + 150MB = 270 MB]

And on some apps (see above), it's a lot more than 600K.
Add them all together, and what do we get? A ton of poop.

Many thousands of unnecessary files being downloaded by millions of users,
cataloged in their HD's disk directory, and needlessly backed up and stored.

Do whatever you feel is right...

Top
#443788 - 06/20/08 06:41 AM Re: jnk ::: identify or erase useless nibs and pli [Re: Hal Itosis]
CharlesS Offline
MacReporter

Registered: 07/25/00
Posts: 368
 Originally Posted By: Hal Itosis
 Originally Posted By: CharlesS
The Keychain, for instance, relies on code-signing to remember whether an app has permission to access it or not. You're going to have weird little things happen, and you'll have a worse computing experience overall,

That's more spin than substance.

First of all, I didn't invent this idea. Apps (linked to above) have been around since Leopard debuted,

And they've always been a bad idea. Some of those apps do even worse things, like deleting all the PPC code out of universal binaries so that Rosetta stops working and you have to reinstall the OS to fix it.

 Quote:
and yes, the keychain may also get confused (if it's being used). So... we just reauthorize (or restart)
and get on with it.

And reauthorize again... and again... and again.

Every time some little thing changes, an app with an invalid signature will have to re-authorize, because the system knows it's been tampered with and no longer trusts it.

You'll also break any other features that turn out to rely on code signing in some way, which I expect to become plentiful as Apple expects all new code to be code-signed. Here's what Apple has to say on this, by the way, in their developer documentation:

Because the system will expect all code to be signed, any code that is not signed will not behave in the same manner as the majority of the programs on the user’s system. In particular, the user is likely to be bothered with additional dialog boxes and prompts for unsigned code that they don’t see with signed code, and unsigned code might not work as expected with some system components, such as parental controls. It is highly recommended that you sign all code intended for use with Mac OS X v10.5 or later.

Therefore, if you are delivering, or intend to deliver, code that might ever be run on Mac OS X v10.5 or later, you should read this document.

 Quote:
 Originally Posted By: CharlesS
and because the extra files only total about 600-700 KB anyway, which is trivial on today's storage devices. That amount would fit on a double-density floppy disk, for crying out loud.

Floppy disk? \:\/

Yep, the non-HD kind, the kind that used to be used in the mid-to-late '80s. Such a massive amount of space, no?

 Quote:
The useless nib files

Just because you don't understand what something does doesn't make it useless. I already explained that without those files, the nibs can't be opened.

 Quote:
in the last 2 Leopard updates were probably bigger than
System 7, System 8 and Mac OS 9 put together. [120MB + 150MB = 270 MB]

Why stop there when comparing to ancient operating systems? Hell, the Calculator app in OS X is about 13 times the size of my old System 6.0.5 installation. System 6.0.5, in turn, dwarfed anything you'd ever find on a Commodore 64. That's progress for ya.

Here's another fun fact - Leopard's icon format is larger in resolution than the size of the original Macintosh's screen.
 Quote:
And on some apps (see above), it's a lot more than 600K.
Add them all together, and what do we get?

About 120 MB, apparently. Most people these days have hard drives that can be measured in 3-digit numbers of gigabytes. 120 MB is nothing - certainly not enough to risk messing up your system over. Hell, my virtual memory swap files are 3 GB right now. Oh well.

Top
#443796 - 06/20/08 08:08 AM Re: jnk ::: identify or erase useless nibs and pli [Re: CharlesS]
MacManiac Moderator Offline
Moderator

Registered: 09/18/01
Posts: 6017
Loc: San Diego, CA (that's my story...
Ted Landau said it a long time ago, but it still relates today....

 Quote:
please limit replies to supplemental suggestions about the initial tip. This is not a Forum for extended discussions and debates.


Hal has provided a worthwhile tip (with issues). Charles has provided detailed responses to those issues. We've got a really good discussion going here, but it's in the wrong forum. I will be moving the discussion portion of this thread into the Lounge where I hope it continues to develop.
_________________________
  • MacFixit Forums Moderator

Top
#443811 - 06/20/08 10:15 AM Re: jnk ::: identify or erase useless nibs and pli [Re: CharlesS]
Hal Itosis Offline
MacWizard

Registered: 08/23/99
Posts: 7032
Loc: 10.5.7 (build 9J61)
 Originally Posted By: CharlesS
Just because you don't understand what something does doesn't make it useless. I already explained that without those files, the nibs can't be opened.


Horsefeathers.     How can you be so disingenuous?

I already pointed out: you can get that info to your 4 friends
(French, German, Italian, and Japanese) by **other** means.
Release versions do not need those nibs... and you KNOW it.

Try to debate this honestly... if at all possible.


--


 Originally Posted By: CharlesS
And reauthorize again... and again... and again.

That's not an accurate picture.

Only Pacifist (and perhaps one or two other apps here and there) are doing this now.
So... you've gone into the future. Fine. But for now, what you said is only 0.01% true.


 Originally Posted By: CharlesS
Every time some little thing changes, an app with an invalid signature will have to re-authorize, because the system knows it's been tampered with and no longer trusts it.

Stop exaggerating. What you say totally misrepresents what's going on here.
There will be a one-time change, and all this authorizing you talk up only
applies to apps that are involved with keychain or firewall (or apps like yours,
who ship dirty because they don't know any better, or simply don't care).



 Originally Posted By: CharlesS
You'll also break any other features that turn out to rely on code signing in some way, which I expect to become plentiful as Apple expects all new code to be code-signed.

That expectation is off in the future.

Hopefully as users become aware of the cruft I'm trying to help them clean,
perhaps utilities like mine won't be needed... because developers like you
might show enough consideration to do your *own* housekeeping.

We will see what eventually happens.

Meanwhile, you do what you think is "right" and I'll do the same.


Edited by Hal Itosis (06/20/08 10:35 AM)

Top
#443819 - 06/20/08 11:28 AM Re: jnk ::: identify or erase useless nibs and pli [Re: Hal Itosis]
MacManiac Moderator Offline
Moderator

Registered: 09/18/01
Posts: 6017
Loc: San Diego, CA (that's my story...
Unless there are further specific and concise additions to this topic relating DIRECTLY to the hint / tip itself, you can find any further discussion **HERE** in the Lounge.
_________________________
  • MacFixit Forums Moderator

Top
Page 1 of 2 1 2 >


Moderator:  dkmarsh, MacManiac 

VersionTracker: Software Updates and Downloads | iPhone Atlas: iPhone Help, News, Tutorials, and Tips