mag_noaa $SMEI/ucsd/gen/python/mag_noaa.py
[Previous] [Next]
 NAME:
	mag_noaa
 PURPOSE:
	Downloads daily-updated photospheric magnetic maps from NOAA and
	run's it through X's HCSS code to make a viable source surface map.
 CALLING SEQUENCE:
	mag_noaa.py <source> <destination> --ftp --sync --gzip --observatory=observatory
	as cron job:
	bash --login -c "mag_noaa.py"
 OPTIONAL INPUTS:
	<source>	source of photospheric magnetic field maps
			This is either a directory containing wso*.dat maps,
			or a text file with a list of fully-qualified names
			of wso*.dat files.
			Default: $DAT/map/<observatory>
	<destination>	destination directory for source surface maps
	--ftp		synchronizes files on NOAA ftp server with local files.
			If -ftp is set then <source>=$DAT/map/wso_noaa and
			<destination>=$DAT/map/wso_noaa/hcss are used, unless
			-source and/or -destination are specified.
	--sync		only processes photospheric maps that do not have
			a matching source surface file in <destination>
	--gzip		gzip the output files in <destination>
	--observatory <observatory>
			observatory; must be one of wso, nso, mwo, nso_ktpk
			default is nso_ktpk (NOAA Solis data)
 CALLS:
	mirror, xhcs
 PROCEDURE:
	Currently  magnetic source surface map at 15 solar radii are made.
 MODIFICATION HISTORY:
	APR-2003, Paul Hick (UCSD/CASS)
	OCT-2003, Paul Hick (UCSD/CASS)
		Added mag_noaa_badfiles to exclude files stored in the text
		file 'badfiles_'+obs+'.txt' located in the destination directory.
	NOV-2003, Paul Hick (UCSD/CASS)
		Added a check to detect when a photospheric map downloaded from
		NOAA overwrites an existing file. In this case the source surface
		file in the destination directory (produced from the overwritten
		file) is deleted to make sure that the new file is processed by
		Xuepus program.
	OCT-2005, Paul Hick (UCSD/CASS)
		Modified default for destination directory to subdirectory
		hcss in source directory.
	JUN-2006, Paul Hick (UCSD/CASS)
		nso_ktpk are now downloaded from remote subdirectory 2006
	FEB-2006, Paul Hick (UCSD/CASS)
		nso_ktpk are now downloaded from remote subdirectory 2007
	JUN-2013, Paul Hick (UCSD/CASS; pphick@ucsd.edu)
		Substantial rewrite to accomodate Solis files from
		Gong pipeline.


mag_noaa_diff $SMEI/ucsd/gen/python/mag_noaa.py
[Previous] [Next]
 NAME:
	mag_noaa_diff
 PURPOSE:
	Looks for entries in array 'lst' that are not present in array 'ref'
 INPUTS:
	prefix_lst
	postfix_lst
	lst
	prefix_ref
	postfix_ref
	ref
 OUTPUTS:
	lst_modified
 PROCEDURE:
	'lst' contains filenames with names (after stripping extensions)
	match prefix_lst at the start and and postfix_lst at the end
	of the name.

	'ref' contains filenames with names (after stripping extensions)
	match prefix_ref at the start and and postfix_ref at the end
	of the name.

	The center part of the filename in between the prefix and postfix is
	used to determine a match: if the center part of a file in 'lst'
	matches the center part of a file in 'ref' it is dropped from the
	the return list.

	The center part of the filenames is a Carrington rotation
	and a Carrington longitude in the form NNNN_LLL with NNNN a 4-digit
	Carrinton rotation and LLL a 3-digit longitude.
	'lst' is a list of solar surface synoptic maps.
	'ref' is a list of hcss synoptic maps.

	If after removing pre- and postfixes from the filenames the
	rotation/longitude in an 'lst' entry is NOT present in the 'ref'
	list than the 'lst' entry represents a solar surface map that
	has not yet been run through Xuepus hcss program.


mirror $SMEI/ucsd/gen/python/mirror.py
[Previous] [Next]
 NAME:
	mirror
 PURPOSE:
	Simple interface to Perl script 'mirror'
 CATEGORY:
	smei/gen/python
 CALLING SEQUENCE:
	From command line:

	mirror.py job=<job> local_dir=<local_dir> remote_dir=<remote_dir>
		remote_site=<remote_site> remote_user=<remote_user> remote_password=<remote_password>
		recursive=<true|false> recurse_hard=<true|false> get_patt=<get_patt>

	As module:

	sts = mirror(job, local_dir, remote_site, remote_dir, remote_user,
		remote_password, recursive, recurse_hard)
 OPTIONAL INPUT PARAMETERS:
	<job>			default: 'job'
				job name (used in the name of the temporary
				mirror package file in $TUB)
	<local_dir>		default: environment variable 'SMEI'
				local directory where the mirror copy is maintained
				The default is the location $SMEI of the local copy
				of the SMEI master tree.
	<remote_site>		default: 'ips.ucsd.edu'
				remote server
	<remote_dir>		default: '/loc_smei'
				directory tree on remote site to be mirrored
				The default is a copy of the SMEI master accessible
				by anonymous ftp only.
	<remote_user>		default: 'anonymous'
	<remote_password>	default: '$USER@$HOSTNAME'
				user and password used by the mirror script to
				log into the ftp server.
				if remote_password='netrc' then an attempt is made
				to obtain the password from $HOME/.netrc
	recursive   =<true|false>
				default: true
	recurse_hard=<true|false>
				default: true (if recursive is set)
				false (if recursive is not set)
	passive_ftp=<true|false>
				default: false
	get_patt = <pattern>
	local_ignore = <local_ignore>
	-save			if set the package file created for mirror.pl is not deleted
 OUTPUTS:
	sts			return status of mirror module
				= None : if mirror was aborted. This currently occurs
				only if mirror is run on ips.
				= [list] : list of files downloaded (this list may be empty!!)

	<local_dir> updated by mirroring <remote_dir>. The results are appended
	to the log file $TUB/<job>.log
 CALLS:
	mirror
 CALLED BY:
	mag_noaa, mirror, sync_ace, sync_ips_mirror [1], sync_ips_mirror [2]
 RESTRICTIONS:
	The following environment variables need to be defined:
	SMEI	points to the local copy of the SMEI master tree
	SSW	points to the root of the SolarSoft tree
		(the Perl script mirror is part of SolarSoft).
 PROCEDURE:
	Location of Perl script 'mirror' and name of
	mirror package file used to mirror the remote directory.
	(the package file is created on the fly and is deleted again).
 MODIFICATION HISTORY:
	DEC-2002, Paul Hick (UCSD/CASS)
		Finalized original version
	DEC-2002, Paul Hick (UCSD/CASS)
		Added a check to avoid overwriting the SMEI master tree
		on ips.ucsd.edu
	MAR-2003, Paul Hick (UCSD/CASS)
		Modified to allow mirror to run as a module.
	NOV-2003, Paul Hick (UCSD/CASS)
		Instead of return 0 for failure and 1 for success, the return
		value is now None for failure and a list of downloaded files for
		success.
	DEC-2003, Paul Hick (UCSD/CASS)
		Changed default of recurse_hard to false if recursive is not set.
		It looks like recursive=false is ignored if recurse_hard=true (the
		previous default) overrides.
	JAN-2004, Paul Hick (UCSD/CASS)
		Added check for existence of Perl script mirror.
		Added check for presence of Perl using first line of mirror script.
	OCT-2005, Paul Hick (UCSD/CASS)
		Added local_ignore keyword.
	NOV-2005, Paul Hick (UCSD/CASS; pphick@ucsd.edu)
		Added passive_ftp keyword.


mk_wind_mag.sh $SMEI/com/linux/mk_wind_mag.sh
[Previous] [Next]
 NAME:
	mk_wind_mag.sh
 PURPOSE:
	Convert downloaded monthly Wind files into
	yearly files needed by IDL insitu software
 CALLING SEQUENCE:
	mk_wind_mag.sh                 (re)create all annual files
	mk_wind_mag.sh -y <YEAR>       (re)create all annual files starting at <YEAR>
	mk_wind_mag.sh -y `date +%Y`   (re)create annual file for current year
 INCLUDE:
	get_options.sh, tiny_bits.sh
 PROCEDURE:
	The monthly files are downloaded by a cronjob in to the
	directory $DAT/insitu/wind_mag. These files are concatenated
	into yearly files $DAT/insitu/windb_<year>.hravg.
 MODIFICATION HISTORY:
	JUL-2013, Paul Hick (UCSD/CASS; pphick@ucsd.edu)


mkramdisk $SMEI/com/linux/mkramdisk
[Previous] [Next]
 NAME:
	mkramdisk
 PURPOSE:
	Set up ramdisk
 CATEGORY:
 CALLING SEQUENCE:
	. $COM/mkramdisk
 INPUTS:
	(ramdisk entry in /etc/fstab)
 OUTPUTS:
	(env var $ramdisk)
 CALLED BY:
	login
 PROCEDURE:
	The ramdisk should be set up already at mount point /tmp/ramdisk.
	This can be done as follows:
		mkdir /tmp/ramdisk
		mke2fs /tmp/ramdisk
	Add the following line to /etc/fstab
		/dev/ramdisk /tmp/ramdisk ext2 noauto,user 0 0

	The ramdisk is primarily used by the IDL procedure gunzip_file
	to store temporary unzipped files.
 MODIFICATION HISTORY:
	JUL-2004, Paul Hick (UCSD/CASS; pphick@ucsd.edu)