soundgen (version 1.7.0)

transplantFormants: Transplant formants

Description

Takes the general spectral envelope of one sound (donor) and "transplants" it onto another sound (recipient). For biological sounds like speech or animal vocalizations, this has the effect of replacing the formants in the recipient sound while preserving the original intonation and (to some extent) voice quality. Note that freqWindow_donor and freqWindow_recipient are crucial parameters that regulate the amount of spectral smoothing in both sounds. The default is to set them to the estimated median pitch, but this is time-consuming and error-prone, so set them to reasonable values manually if possible. Also ensure that both sounds have the same sampling rate.

Usage

transplantFormants(
  donor,
  freqWindow_donor = NULL,
  recipient,
  freqWindow_recipient = NULL,
  samplingRate = NULL,
  dynamicRange = 80,
  windowLength = 50,
  step = NULL,
  overlap = 90,
  wn = "gaussian",
  zp = 0
)

Arguments

donor

the sound that provides the formants or the desired spectral filter as returned by getSpectralEnvelope

freqWindow_donor, freqWindow_recipient

the width of smoothing window. Defaults to median pitch of each respective sound estimated by analyze

recipient

the sound that receives the formants

samplingRate

sampling rate of x (only needed if x is a numeric vector, rather than an audio file)

dynamicRange

dynamic range, dB. All values more than one dynamicRange under maximum are treated as zero

windowLength

length of FFT window, ms

step

you can override overlap by specifying FFT step, ms

overlap

overlap between successive FFT frames, %

wn

window type: gaussian, hanning, hamming, bartlett, rectangular, blackman, flattop

zp

window length after zero padding, points

Details

Algorithm: makes spectrograms of both sounds, interpolates and smoothes the donor spectrogram, flattens the recipient spectrogram, multiplies the spectrograms, and transforms back into time domain with inverse STFT.

See Also

transplantEnv getSpectralEnvelope addFormants soundgen

Examples

# NOT RUN {
# Objective: take formants from the bleating of a sheep and apply them to a
# synthetic sound with any arbitrary duration, intonation, nonlinearities etc
data(sheep, package = 'seewave')  # import a recording from seewave
donor = as.numeric(scale(sheep@left))  # source of formants
samplingRate = [e-mail]
playme(donor, samplingRate)
spectrogram(donor, samplingRate, osc = TRUE)
seewave::meanspec(donor, f = samplingRate, dB = 'max0')

recipient = soundgen(sylLen = 1200,
                     pitch = c(100, 300, 250, 200),
                     vibratoFreq = 9, vibratoDep = 1,
                     addSilence = 180,
                     samplingRate = samplingRate,
                     invalidArgAction = 'ignore')  # keep low samplingRate
playme(recipient, samplingRate)
spectrogram(recipient, samplingRate, osc = TRUE)

s1 = transplantFormants(
  donor = donor,
  recipient = recipient,
  samplingRate = samplingRate)
playme(s1, samplingRate)
spectrogram(s1, samplingRate, osc = TRUE)
seewave::meanspec(s1, f = samplingRate, dB = 'max0')

# if needed, transplant amplitude envelopes as well:
s2 = transplantEnv(donor = donor, samplingRateD = samplingRate,
                   recipient = s1, windowLength = 10)
playme(s2, samplingRate)
spectrogram(s2, samplingRate, osc = TRUE)

# Now we use human formants on sheep source: the sheep says "why?"
s2 = transplantFormants(
  donor = soundgen(formants = 'uaaai',
                   samplingRate = samplingRate,
                   invalidArgAction = 'ignore'),
  recipient = donor,
  samplingRate = samplingRate)
playme(s2, samplingRate)
spectrogram(s2, samplingRate, osc = TRUE)
seewave::meanspec(s2, f = samplingRate, dB = 'max0')

# We can also transplant synthetic formants w/o synthesizing a donor sound to
# save time
s3 = transplantFormants(
  donor = getSpectralEnvelope(
            nr = 512, nc = 100,  # fairly arbitrary dimensions
            formants = 'uaaai',
            samplingRate = samplingRate),
  recipient = donor,
  samplingRate = samplingRate)
playme(s3, samplingRate)
spectrogram(s3, samplingRate, osc = TRUE)
# }