Introduction to 1D Convolutional Neural Networks

Introduction to 1D Convolutional Neural Networks#

Mahmood Amintoosi, Spring 2026

Computer Science Dept, Ferdowsi University of Mashhad

Learning Objectives#

By the end of this chapter, you will:

Understand how 1D convolution works mathematically and intuitively
Apply 1D CNNs to time series data and signal processing tasks
Recognize the relationship between kernel size, receptive field, and smoothing effects
Be prepared to extend these concepts to 2D convolution for images

1. Why 1D Convolution?#

Before diving into image processing with 2D CNNs, let’s understand convolution in one dimension. Many real-world problems involve sequential or time-series data:

Domain	Example Data
Finance	Stock prices, trading volumes over time
Healthcare	ECG signals, heart rate monitoring
Audio	Sound waves, speech signals
IoT/Sensors	Temperature readings, accelerometer data
Natural Language	Text sequences (though often handled differently)
Traffic Forecasting	Traffic parameters, like speed and volume

1D CNNs are particularly effective for these data types because they can:

Detect local patterns regardless of their position in the sequence
Reduce dimensionality while preserving important features
Share parameters across the entire sequence

2. Mathematical Foundation of 1D Convolution#

2.1 The Convolution Operation#

Given an input signal \(x\) of length \(n\) and a kernel (filter) \(w\) of length \(k\), the convolution operation produces output \(y\):

\[y[i] = \sum_{j=0}^{k-1} x[i+j] \cdot w[j]\]

where \(i\) ranges from \(0\) to \(n-k\) (for “valid” convolution).

2.2 Manual Example#

Let’s trace through a simple convolution:

Input x:     [1, 5, 3, 4, 8]
Kernel w:    [1/3, 1/3, 1/3]  (moving average filter)

y[0] = 1×(1/3) + 5×(1/3) + 3×(1/3) = 9/3 = 3.0
y[1] = 5×(1/3) + 3×(1/3) + 4×(1/3) = 12/3 = 4.0
y[2] = 3×(1/3) + 4×(1/3) + 8×(1/3) = 15/3 = 5.0

Output y:    [3.0, 4.0, 5.0]

Notice how the kernel “slides” across the input, computing a weighted sum at each position.

2.3 Different Kernel Effects#

The same input with different kernels produces different results:

Kernel	Effect	Output for [1, 5, 3, 4, 8]
`[1/3, 1/3, 1/3]`	Moving average (smoothing)	`[3.0, 4.0, 5.0]`
`[0.25, 0.5, 0.25]`	Weighted average (center-weighted)	`[3.5, 3.75, 4.75]`
`[-1, 0, 1]`	Edge detection (approximate derivative)	`[2, -1, 4]`
`[1, -2, 1]`	Second derivative (peak detection)	`[-6, 3, 6]`

Exercise: Verify the edge detection kernel output. What pattern do you notice about where the values change significantly?

3. Implementing 1D Convolution#

3.1 Using SciPy#

import numpy as np
from scipy import signal

# Define input and kernel
x = np.array([1, 5, 3, 4, 8])
w = np.array([1/3, 1/3, 1/3])

# Apply convolution
y = signal.convolve(x, w, mode="valid")
print(f"Input:  {x}")
print(f"Output: {y}")

Input:  [1 5 3 4 8]
Output: [3. 4. 5.]

Mode options:

"valid": Returns only positions where kernel fully overlaps input (size: \(n-k+1\))
"same": Returns same length as input with zero-padding
"full": Returns all positions including partial overlaps (size: \(n+k-1\))

3.2 Using PyTorch#

PyTorch expects input tensors with shape (batch_size, channels, length):

import torch
import torch.nn as nn
import numpy as np

# Reshape input: (samples, channels, length)
x = np.array([1, 5, 3, 4, 8])
x_tensor = torch.from_numpy(x).float().view(1, 1, 5)

# Define Conv1d layer
conv1d = nn.Conv1d(in_channels=1, out_channels=1, kernel_size=3, bias=False)

# Initialize with moving average weights
w = torch.tensor([1/3, 1/3, 1/3]).view(1, 1, 3)
conv1d.weight.data = w

# Apply convolution
y_tensor = conv1d(x_tensor)
print(y_tensor)
# tensor([[[3., 4., 5.]]], grad_fn=<ConvolutionBackward0>)

tensor([[[3., 4., 5.]]], grad_fn=<ConvolutionBackward0>)

Key parameters:

in_channels: Number of input features (e.g., 1 for univariate time series)
out_channels: Number of filters (output features)
kernel_size: Length of the convolution window
stride: Step size between positions (default: 1)
padding: Zeros added to both sides (default: 0)

4. Application: Signal Denoising#

One of the most intuitive applications of 1D convolution is smoothing noisy signals. Let’s explore this with a sine wave.

4.1 Generating Noisy Data#

import numpy as np
import matplotlib.pyplot as plt

# Generate clean sine wave
t = np.linspace(0, 1, 1000)
frequency = 5
clean_signal = np.sin(2 * np.pi * frequency * t)

# Add Gaussian noise
noise = np.random.normal(0, 0.2, clean_signal.shape)
noisy_signal = clean_signal + noise

plt.figure(figsize=(12, 4))
plt.plot(t, noisy_signal, 'b', alpha=0.5, label='Noisy Signal')
plt.plot(t, clean_signal, 'y', linewidth=2, label='Clean Signal')
plt.legend()
plt.title('Noisy vs Clean Sine Wave')
plt.xlabel('Time [s]')
plt.ylabel('Amplitude')
plt.show()

_images/760c47bc30ed05855d92e39f57fe3ef339ffc26199d95107492d0ef490f5a87c.png

4.2 Effect of Kernel Size#

The kernel size determines the “window” of smoothing. Let’s compare:

_images/13cd4533004185c4572a6d61b2f2768b7839f7da1a1fe373a4277ffedb4b1b88.png

Understanding Phase Delay in Convolution#

When we apply a 1D convolution with kernel size \(k\), the output at each position represents a weighted average (or sum) of \(k\) input values. By default, PyTorch’s Conv1d with padding=0 uses “valid” convolution, meaning the kernel only slides where it fully overlaps with the input.

The Phase Delay Phenomenon#

The output value at index \(i\) actually corresponds to information centered around index \(i + \frac{k-1}{2}\) in the original signal. This creates a phase delay (or lag) of approximately \(\frac{k}{2}\) samples.

For example, with kernel size 29:

The output at position 0 uses input samples 0–28
The center of this window is at position 14
Therefore, the output appears to “lag” behind the input by ~14 samples

Visualizing the Effect#

We can demonstrate this by plotting the smoothed signal with and without offset correction:

With offset correction (t[k//2 : k//2 + len(smoothed)]): We manually shift the time axis to align peaks, which is useful for comparison but hides the delay
Without offset correction: The smoothed curve appears shifted to the left (earlier in time), revealing the true phase delay

_images/3e3e9adbed77f447a0a6425a96c3960514fb45b5527fb3b4c5e067b611b624cc.png

Observations:

Small kernel (3): Removes high-frequency noise but preserves most signal detail
Medium kernel (9): Stronger smoothing, some phase shift
Large kernel (29): Very smooth but may blur important features and introduces significant phase delay

4.3 The Trade-off: Smoothing vs Detail Preservation#

Kernel Size	Noise Reduction	Detail Preservation	Phase Shift
Small	Low	High	Minimal
Medium	Medium	Medium	Moderate
Large	High	Low	Significant

Key Insight: The kernel size determines the receptive field—how much of the input each output point can “see.” Larger kernels see more context but lose localization precision.

5. Pattern Detection with Template Matching#

One of the most powerful applications of 1D convolution is template matching—detecting a specific pattern within a longer signal. This is the foundation of many real-world systems:

Application	What We Detect
Speech recognition	Specific phonemes in audio
Seismic monitoring	Earthquake signatures in sensor data
Fault detection	Anomaly patterns in machine vibrations
Music information retrieval	Drum beats or melody fragments

5.1 The Concept: Convolution as Pattern Matching#

When we convolve a signal with a template (kernel), the output peaks where the signal matches the template. This works because:

\[\text{Output}[i] = \sum_{j} \text{Signal}[i+j] \times \text{Template}[j]\]

High values indicate strong similarity; low (or negative) values indicate mismatch.

5.2 Example: Detecting a “Triangle” Pattern in Noise#

Let’s create a simple, visual example: detecting a triangular pulse in a noisy signal.

_images/f05cdb2136e88563f98b4fcb1a7072230204e61a622b95df5ffc4fd48b56905d.png

5.3 Applying Template Matching with Convolution#

Now we use the template as our convolution kernel. The output will peak where the pattern matches:

Show code cell source Hide code cell source

# Apply convolution
conv1d = nn.Conv1d(1, 1, template_length, bias=False)
conv1d.weight.data = torch.tensor(template_normalized).view(1, 1, template_length).float()

signal_tensor = torch.from_numpy(signal).view(1, 1, -1).float()
with torch.no_grad():
    response = conv1d(signal_tensor).numpy().squeeze()

fig, axes = plt.subplots(2, 1, figsize=(12, 8))

axes[0].plot(t, signal, 'b-', alpha=0.7, label='Signal')
axes[0].axvspan(40, 40+template_length, alpha=0.2, color='green', label='Target Pattern (strong)')
axes[0].axvspan(140, 140+template_length, alpha=0.2, color='green', label='Target Pattern (weak)')
axes[0].axvspan(90, 90+len(distractor), alpha=0.2, color='orange', label='Distractor')
axes[0].set_title('Input Signal', fontsize=11)
axes[0].set_ylabel('Amplitude')
axes[0].legend(loc='upper right', fontsize=8)
axes[0].grid(True, alpha=0.3)

# Convolution response
response_t = t[:len(response)]
axes[1].plot(response_t, response, 'g-', linewidth=2, label='Convolution Response')
axes[1].axhline(y=0, color='k', linestyle='-', linewidth=0.5)

# Find peaks with HIGHER threshold to reject distractor
peaks, properties = find_peaks(response, height=0.8, distance=template_length//2)

# Use TRIANGLE marker (▲) for detected peaks - indicates "peak/match"
axes[1].plot(response_t[peaks], response[peaks], 'r^', markersize=12, 
             markeredgewidth=2, markerfacecolor='yellow', 
             markeredgecolor='red', label=f'Detected Matches (n={len(peaks)})')

# Add vertical lines to show alignment
for peak in peaks:
    axes[1].axvline(x=response_t[peak], color='r', linestyle='--', alpha=0.5)
    axes[0].axvline(x=response_t[peak], color='r', linestyle='--', alpha=0.5)

axes[1].set_title('Convolution Output (Template Matching Response)', fontsize=11)
axes[1].set_xlabel('Sample Index')
axes[1].set_ylabel('Match Strength')
axes[1].legend()
axes[1].grid(True, alpha=0.3)

plt.tight_layout()
plt.show()

# Analysis
print(f"\n{'='*60}")
print("TEMPLATE MATCHING RESULTS")
print(f"{'='*60}")
print(f"Template length: {template_length} samples")
print(f"Signal length: {signal_length} samples")
print(f"Output length: {len(response)} samples (valid convolution)")

print(f"\nDetected peaks at positions: {response_t[peaks].astype(int)}")
print(f"Peak values (match strength): {[f'{v:.3f}' for v in response[peaks]]}")

expected_positions = [40, 140]
print(f"\nExpected pattern positions: {expected_positions}")
print(f"Actual detected positions: {list(response_t[peaks].astype(int))}")

# Check detection accuracy
detected_positions = list(response_t[peaks].astype(int))
correct_detections = sum(1 for exp in expected_positions 
                        if any(abs(d - exp) < 5 for d in detected_positions))
print(f"\nCorrect detections: {correct_detections}/{len(expected_positions)}")

# Check distractor rejection
distractor_center = 94
distractor_response = response[distractor_center - 5:distractor_center + 5]
print(f"\nDistractor region response (max): {np.max(distractor_response):.3f}")
print(f"Detection threshold: 0.8")
print(f"Distractor correctly rejected: {np.max(distractor_response) < 0.8}")

# Show all responses for comparison
print(f"\nAll peak responses:")
print(f"  Position 40 (target, amp=0.7):  {response[40]:.3f}")
print(f"  Position 94 (distractor):       {response[distractor_center]:.3f}")
print(f"  Position 140 (target, amp=0.5): {response[140]:.3f}")

_images/0797a79b2cab1427f2aacbf091d62086303c73e856296eae8684d119e11610bd.png

============================================================
TEMPLATE MATCHING RESULTS
============================================================
Template length: 9 samples
Signal length: 200 samples
Output length: 192 samples (valid convolution)

Detected peaks at positions: [ 40 140]
Peak values (match strength): ['1.225', '0.940']

Expected pattern positions: [40, 140]
Actual detected positions: [np.int64(40), np.int64(140)]

Correct detections: 2/2

Distractor region response (max): 0.599
Detection threshold: 0.8
Distractor correctly rejected: True

All peak responses:
  Position 40 (target, amp=0.7):  1.225
  Position 94 (distractor):       0.599
  Position 140 (target, amp=0.5): 0.940

5.4 Why This Works: The Mathematics#

The convolution output at position \(i\) computes the cross-correlation between the normalized template and the signal window starting at \(i\):

\[\text{Response}[i] = \sum_{j} \text{Signal}[i+j] \times \text{Template}[j]\]

High positive value: Signal matches template shape
Near zero: No correlation (random noise)
Negative value: Signal is opposite of template

This is essentially computing the cosine similarity between the template and local signal patches.

5.5 Key Insights#

Observation	Explanation
Two clear peaks	Template detected at both insertion points
Peak heights differ	0.5 amplitude pattern produces weaker response than 0.7
Distractor rejected	Distractor shape doesn’t match triangular template
Peak width	Related to template length; broader templates = broader peaks

5.6 Real-World Connection: From 1D to 2D#

This same principle extends directly to 2D CNNs for images:

1D (This Example)	2D (Next Chapter)
Detect triangular pulse in signal	Detect edges, corners in images
Template: 1D array of length \(k\)	Template: 2D kernel of size \(k \times k\)
Output: 1D response curve	Output: 2D feature map (activation map)
Peaks = pattern locations	Bright spots = pattern locations

Preview: In the next chapter, instead of detecting 1D patterns like triangles, we’ll detect 2D patterns like vertical edges, horizontal edges, and textures in images using Conv2d.

Exercise: Design Your Own Detector#

Modify the template to detect different patterns:

Square wave (detect on/off transitions)
Gaussian pulse (detect smooth bumps)
Derivative operator (detect sudden changes)

Observe how the convolution response changes with each template shape.

6. Multi-Output Convolution: Detecting Multiple Patterns#

In previous sections, we used a single filter to detect one pattern (triangle or sawtooth). Real signals often contain multiple types of patterns. Instead of running separate convolutions, we can use a single layer with multiple output channels—each channel acts as an independent detector.

6.1 The Setup: A Signal with Two Different Patterns#

We’ll create a signal containing:

Pattern A: A triangular pulse (appears at position 60)
Pattern B: A rectangular pulse (appears at position 220)

Both are “events” but have different shapes. We want to detect each type separately.

_images/15917e5f2865040a36da3925da145d73cd2af6748b2d969b58268e53c2ef3be7.png

_images/1adc155f6a6f232fed17abcd2205b5c0cc047382dbe979045f1d4c078b41cd0d.png

6.2 Creating a Two-Channel Detector#

Now we create ONE convolution layer with TWO output channels:

Channel 0: Detects triangles
Channel 1: Detects rectangles

# Normalize both templates
tri_norm = template_triangle - np.mean(template_triangle)
tri_norm = tri_norm / np.linalg.norm(tri_norm)

rect_norm = template_rectangle - np.mean(template_rectangle)
rect_norm = rect_norm / np.linalg.norm(rect_norm)

# Pad to same length (needed for Conv1d)
max_length = max(len(tri_norm), len(rect_norm))
tri_padded = np.zeros(max_length)
tri_padded[:len(tri_norm)] = tri_norm

rect_padded = np.zeros(max_length)
start_idx = (max_length - len(rect_norm)) // 2
rect_padded[start_idx:start_idx+len(rect_norm)] = rect_norm

# Stack as two filters: shape (2, 1, max_length)
# Dimension 0 = output channels (2 filters)
# Dimension 1 = input channels (1 signal)
# Dimension 2 = kernel size
weights = np.stack([tri_padded, rect_padded])

print(f"Weights shape: {weights.shape}")
print(f"  → 2 output channels (one per pattern)")
print(f"  → 1 input channel (single signal)")
print(f"  → kernel size {max_length}")

# Create Conv1d with 1 input → 2 output channels
conv_detector = nn.Conv1d(
    in_channels=1,      # One input signal
    out_channels=2,     # Two detectors (triangle + rectangle)
    kernel_size=max_length,
    bias=False
)

# Load our hand-designed weights
conv_detector.weight.data = torch.tensor(weights).float().view(2, 1, max_length)

print(f"\nConv1d weight shape: {conv_detector.weight.shape}")
print(f"  [0, 0, :] = Triangle detector")
print(f"  [1, 0, :] = Rectangle detector")

Weights shape: (2, 9)
  → 2 output channels (one per pattern)
  → 1 input channel (single signal)
  → kernel size 9

Conv1d weight shape: torch.Size([2, 1, 9])
  [0, 0, :] = Triangle detector
  [1, 0, :] = Rectangle detector

6.3 Applying and Visualizing Detection#

Show code cell source Hide code cell source

# Prepare input
signal_tensor = torch.from_numpy(signal).view(1, 1, -1).float()

# Apply convolution
with torch.no_grad():
    output = conv_detector(signal_tensor)

# Extract both channels
response_triangle = output[0, 0, :].numpy()     # Channel 0
response_rectangle = output[0, 1, :].numpy()   # Channel 1

# Adjust time axis for valid convolution
offset = max_length // 2
t_out = t[offset:offset+len(response_triangle)]

fig, axes = plt.subplots(3, 1, figsize=(12, 10))

# Input signal
axes[0].plot(t, signal, 'g-', alpha=0.7)
axes[0].axvspan(60, 60+pattern_length, alpha=0.2, color='red')
axes[0].axvspan(220, 220+pattern_length, alpha=0.2, color='blue')
axes[0].set_title('INPUT SIGNAL', fontsize=11)
axes[0].set_ylabel('Amplitude')
axes[0].grid(True, alpha=0.3)

# Channel 0: Triangle detector
axes[1].plot(t_out, response_triangle, 'r-', linewidth=2, label='Channel 0 Output')
from scipy.signal import find_peaks
peaks_tri, _ = find_peaks(response_triangle, height=0.8, distance=max_length//2)
axes[1].plot(t_out[peaks_tri], response_triangle[peaks_tri], 'r^', markersize=12,
             markerfacecolor='yellow', markeredgecolor='red', markeredgewidth=2,
             label=f'Detected Triangles (n={len(peaks_tri)})')
axes[1].set_title('CHANNEL 0: Triangle Pattern Detector', fontsize=11, color='red', fontweight='bold')
axes[1].set_ylabel('Match Strength')
axes[1].legend()
axes[1].grid(True, alpha=0.3)

# Channel 1: Rectangle detector
axes[2].plot(t_out, response_rectangle, 'b-', linewidth=2, label='Channel 1 Output')
peaks_rect, _ = find_peaks(response_rectangle, height=0.8, distance=max_length//2)
axes[2].plot(t_out[peaks_rect], response_rectangle[peaks_rect], 'bs', markersize=12,
             markerfacecolor='yellow', markeredgecolor='blue', markeredgewidth=2,
             label=f'Detected Rectangles (n={len(peaks_rect)})')
axes[2].set_title('CHANNEL 1: Rectangle Pattern Detector', fontsize=11, color='blue', fontweight='bold')
axes[2].set_xlabel('Sample Index')
axes[2].set_ylabel('Match Strength')
axes[2].legend()
axes[2].grid(True, alpha=0.3)

plt.tight_layout()
plt.show()

# Analysis
print(f"\n{'='*60}")
print("MULTI-PATTERN DETECTION RESULTS")
print(f"{'='*60}")
print(f"\nChannel 0 (Triangle Filter):")
print(f"  Detected at positions: {t_out[peaks_tri].astype(int)}")
print(f"  Expected: 60")
print(f"  Peak strength: {response_triangle[peaks_tri][0]:.3f}")

print(f"\nChannel 1 (Rectangle Filter):")
print(f"  Detected at positions: {t_out[peaks_rect].astype(int)}")
print(f"  Expected: 220")
print(f"  Peak strength: {response_rectangle[peaks_rect][0]:.3f}")

print(f"\nKey Observation:")
print(f"  Each channel responds ONLY to its specific pattern")
print(f"  Triangle filter ignores rectangles, rectangle filter ignores triangles")

_images/a556493a7e564309b79f1e09d64c4b70e29fd3564296138ab6f70ca636b9c8d0.png

============================================================
MULTI-PATTERN DETECTION RESULTS
============================================================

Channel 0 (Triangle Filter):
  Detected at positions: [ 64 227]
  Expected: 60
  Peak strength: 1.302

Channel 1 (Rectangle Filter):
  Detected at positions: [224]
  Expected: 220
  Peak strength: 1.044

Key Observation:
  Each channel responds ONLY to its specific pattern
  Triangle filter ignores rectangles, rectangle filter ignores triangles

6.4 What This Means for Deep Learning#

Concept	In This Example	In Deep CNNs
Output channels	2 (triangle, rectangle)	64, 128, 256…
What each channel detects	Hand-designed pattern	Learned automatically
How weights are set	We designed them	Backpropagation learns them

The big idea: Instead of us designing triangle/rectangle detectors, a deep network learns what to detect. Early layers might learn edges, middle layers learn shapes, late layers learn objects.

Summary#

Stage	What We Did	Channels
Section 5	Detect one pattern (triangle)	1 → 1
This section	Detect two patterns (triangle + rectangle)	1 → 2
Next (Section 7)	Full classifier with many channels	1 → 16 → 32 → 64

Key takeaway: Multiple output channels = multiple detectors working in parallel. In deep learning, these detectors are learned, not hand-designed.

7. From Pattern Detection to Frequency Filtering#

So far, we’ve used convolution to detect specific shapes (triangles and rectangles). Another powerful use of convolution is frequency filtering—separating slow-varying components from rapid changes.

7.1 Intuition: What’s “Slow” vs “Fast”?#

Signal Type	Visual Description	Example
Slow (Low Frequency)	Changes gradually over many samples	Baseline drift, trends
Fast (High Frequency)	Changes rapidly, many oscillations	Noise, sharp edges, fine details

Think of a temperature sensor:

Slow: Daily temperature cycle (rises in morning, falls at night)
Fast: Random measurement noise (jumps between consecutive readings)

7.2 The Moving Average: A Simple Low-Pass Filter#

A low-pass filter lets slow components pass through and blocks fast components. The simplest version is the moving average:

_images/f710fe29c01339b9761bed7c2257f12497583396d534b68195ac4803b58401ed.png

What happened?

The moving average computes the local mean over 21 samples
Fast oscillations cancel out (positive and negative values average to ~0)
Slow variations survive because they don’t change much within the window

7.3 Why Kernel Size Matters#

Kernel Size	Effect	Output
Small (3)	Weak smoothing, some fast components remain	Closer to original
Medium (11)	Moderate smoothing	Balanced
Large (51)	Strong smoothing, only very slow components remain	Very smooth

_images/f06b8173e57ba3491b269668e214cef7ffcb37f3024b90cd29fb0159cf62fdcd.png

7.4 High-Pass Filter: Keeping Only Fast Changes#

A high-pass filter does the opposite: it removes slow components and keeps rapid changes.

The trick: Original − Smoothed = Detail

_images/9d3e8e3e7a9397c6109e61d9889223156186bd9d2f1dfa03dbcfc52e6e5c6f0d.png

7.5 Connection to Pattern Detection#

You might notice: these filters look like “patterns” too!

Filter	Kernel Shape	What It Detects
Triangle detector	Triangle shape	Specific pattern at specific location
Moving average	Flat/constant	Slow, unchanging regions
High-pass (derivative)	`[-1, 0, 1]`	Rapid changes (edges)

The difference:

Pattern detection: Kernel = shape we’re looking for
Frequency filtering: Kernel = mathematical operation that selects speed of variation

Note on Filter Design

In this educational example, we hand-designed the filters using prior knowledge of the signal components.

In real-world scenarios where components are unknown, we rely on:

Learned filters in CNNs: Network discovers optimal filters through backpropagation

Blind source separation (BSS): ICA, NMF, or autoencoders separate mixed signals without templates

Adaptive filtering: Algorithms like LMS/RLS adjust filters dynamically

Further Reading: Blind Deconvolution, Independent Component Analysis (ICA), and ARIMA models for time series decomposition.

8. Building a 1D CNN Classifier#

Let’s build a complete 1D CNN for classifying different types of signals.

8.1 Architecture#

class SignalClassifier(nn.Module):
    def __init__(self, input_channels=1, num_classes=3):
        super(SignalClassifier, self).__init__()
        
        # Layer 1: Extract low-level features (edges, basic shapes)
        self.conv1 = nn.Conv1d(input_channels, 16, kernel_size=7, padding=3)
        self.bn1 = nn.BatchNorm1d(16)
        self.relu1 = nn.ReLU()
        self.pool1 = nn.MaxPool1d(2)
        
        # Layer 2: Extract higher-level patterns
        self.conv2 = nn.Conv1d(16, 32, kernel_size=5, padding=2)
        self.bn2 = nn.BatchNorm1d(32)
        self.relu2 = nn.ReLU()
        self.pool2 = nn.MaxPool1d(2)
        
        # Layer 3: Deep features
        self.conv3 = nn.Conv1d(32, 64, kernel_size=3, padding=1)
        self.bn3 = nn.BatchNorm1d(64)
        self.relu3 = nn.ReLU()
        self.pool3 = nn.AdaptiveAvgPool1d(1)  # Global average pooling
        
        # Classifier
        self.fc = nn.Linear(64, num_classes)
        
    def forward(self, x):
        # Feature extraction
        x = self.pool1(self.relu1(self.bn1(self.conv1(x))))
        x = self.pool2(self.relu2(self.bn2(self.conv2(x))))
        x = self.pool3(self.relu3(self.bn3(self.conv3(x))))
        
        # Flatten and classify
        x = x.view(x.size(0), -1)
        x = self.fc(x)
        return x

# Create model
model = SignalClassifier(input_channels=1, num_classes=3)
print(model)

# Test with dummy input
test_input = torch.randn(4, 1, 1000)  # Batch of 4 signals
output = model(test_input)
print(f"\nInput shape: {test_input.shape}")
print(f"Output shape: {output.shape}")

SignalClassifier(
  (conv1): Conv1d(1, 16, kernel_size=(7,), stride=(1,), padding=(3,))
  (bn1): BatchNorm1d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (relu1): ReLU()
  (pool1): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (conv2): Conv1d(16, 32, kernel_size=(5,), stride=(1,), padding=(2,))
  (bn2): BatchNorm1d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (relu2): ReLU()
  (pool2): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  (conv3): Conv1d(32, 64, kernel_size=(3,), stride=(1,), padding=(1,))
  (bn3): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (relu3): ReLU()
  (pool3): AdaptiveAvgPool1d(output_size=1)
  (fc): Linear(in_features=64, out_features=3, bias=True)
)

Input shape: torch.Size([4, 1, 1000])
Output shape: torch.Size([4, 3])

8.2 Training Loop Structure#

import torch.optim as optim

# Setup
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)

# Training step (simplified)
def train_step(model, data, labels):
    optimizer.zero_grad()
    outputs = model(data)
    loss = criterion(outputs, labels)
    loss.backward()
    optimizer.step()
    return loss.item()

8.3 Application: Learn the Moving Average Kernel#

Task: Learn the kernel weights through training instead of hand-designing.

import torch
import torch.nn as nn
import torch.optim as optim
import numpy as np

# Target: moving average smoothing
def moving_average(x, kernel_size=3):
    kernel = np.ones(kernel_size) / kernel_size
    result = np.convolve(x, kernel, mode='valid')
    return result

# Generate data: random signals → smoothed versions
np.random.seed(42)
n_samples = 500
signal_length = 30

X = []
y = []

for _ in range(n_samples):
    # Random signal
    sig = np.random.randn(signal_length)
    # Target: smoothed version
    smoothed = moving_average(sig)
    
    X.append(sig)
    y.append(smoothed)


X = torch.tensor(X).float().view(n_samples, 1, signal_length)
print(X.shape)
y = torch.tensor(y).float().view(n_samples, 1, signal_length - 2)  # valid conv reduces length

# Simple model: single conv layer
class LearnableSmoothing(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv = nn.Conv1d(1, 1, kernel_size=3, bias=False)
        
    def forward(self, x):
        return self.conv(x)

model = LearnableSmoothing()
criterion = nn.MSELoss()
optimizer = optim.Adam(model.parameters(), lr=0.01)

# Train
print("Initial kernel:", model.conv.weight.data.squeeze().numpy())
print("Target kernel: [0.333, 0.333, 0.333]")

for epoch in range(100):
    optimizer.zero_grad()
    output = model(X)
    loss = criterion(output, y)
    loss.backward()
    optimizer.step()
    
    if epoch % 20 == 0:
        print(f"Epoch {epoch}, Loss: {loss.item():.6f}")
        print(f"  Learned kernel: {model.conv.weight.data.squeeze().detach().numpy()}")

print(f"\nFinal learned kernel: {model.conv.weight.data.squeeze().detach().numpy()}")
print(f"Target kernel:        [0.333, 0.333, 0.333]")

C:\Users\m.amintoosi\AppData\Local\Temp\ipykernel_14464\2001567032.py:30: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at C:\actions-runner\_work\pytorch\pytorch\builder\windows\pytorch\torch\csrc\utils\tensor_new.cpp:281.)
  X = torch.tensor(X).float().view(n_samples, 1, signal_length)

torch.Size([500, 1, 30])
Initial kernel: [ 0.23811197 -0.44154182 -0.10843295]
Target kernel: [0.333, 0.333, 0.333]
Epoch 0, Loss: 0.798964
  Learned kernel: [ 0.24811196 -0.43154183 -0.09843295]
Epoch 20, Loss: 0.395849
  Learned kernel: [ 0.34302208 -0.2356088   0.09336054]
Epoch 40, Loss: 0.171167
  Learned kernel: [ 0.3243526  -0.06021212  0.24031174]
Epoch 60, Loss: 0.067036
  Learned kernel: [0.3293367  0.08185966 0.31694666]
Epoch 80, Loss: 0.023052
  Learned kernel: [0.33287185 0.18639144 0.33751038]

Final learned kernel: [0.3325661  0.25310472 0.3359231 ]
Target kernel:        [0.333, 0.333, 0.333]

9. Application: ECG Heartbeat Detection (Further Reading)#

Let’s apply 1D CNN concepts to a more realistic problem: detecting heartbeats in ECG data.

9.1 Understanding ECG Signals#

An ECG (electrocardiogram) measures electrical activity of the heart. Key features include:

P wave: Atrial depolarization
QRS complex: Ventricular depolarization (strongest signal)
T wave: Ventricular repolarization

9.2 Simulating ECG Data#

def generate_ecg_signal(length=1000, heart_rate=60, noise_level=0.05):
    """
    Generate synthetic ECG-like signal
    """
    t = np.linspace(0, 10, length)
    beat_interval = 60 / heart_rate  # seconds per beat
    num_beats = int(10 / beat_interval)
    
    signal = np.zeros(length)
    for i in range(num_beats):
        beat_time = i * beat_interval
        beat_idx = int(beat_time / 10 * length)
        
        # Create QRS complex (simplified)
        if beat_idx < length - 20:
            # P wave
            signal[beat_idx-10:beat_idx-5] += 0.1 * np.sin(np.linspace(0, np.pi, 5))
            # QRS complex
            signal[beat_idx:beat_idx+3] += -0.3  # Q
            signal[beat_idx+3:beat_idx+6] += 1.0  # R (peak)
            signal[beat_idx+6:beat_idx+9] += -0.2  # S
            # T wave
            signal[beat_idx+12:beat_idx+20] += 0.15 * np.sin(np.linspace(0, np.pi, 8))
    
    # Add noise
    signal += np.random.normal(0, noise_level, length)
    return t, signal

# Generate signal
t, ecg = generate_ecg_signal()

plt.figure(figsize=(12, 4))
plt.plot(t, ecg)
plt.title('Synthetic ECG Signal')
plt.xlabel('Time (s)')
plt.ylabel('Amplitude')
plt.grid(True, alpha=0.3)
plt.show()

_images/359882d617163d7727e823756bc11f42bb77ed44e8e80c6253f7f201747bef00.png

9.3 Detecting R-peaks with Convolution#

We can design a kernel that responds strongly to QRS complexes:

# Create a matched filter kernel (template matching)
template_length = 15
template = np.zeros(template_length)
template[3:6] = -0.3   # Q
template[6:9] = 1.0    # R
template[9:12] = -0.2  # S

# Normalize template
template = template / np.sum(template**2)

# Apply convolution
conv1d = nn.Conv1d(1, 1, template_length, bias=False)
conv1d.weight.data = torch.tensor(template).view(1, 1, template_length).float()

ecg_tensor = torch.from_numpy(ecg).view(1, 1, -1).float()
response = conv1d(ecg_tensor).detach().numpy().squeeze()

# Plot results
fig, axes = plt.subplots(2, 1, figsize=(12, 6))

axes[0].plot(t, ecg, 'b', label='ECG Signal')
axes[0].set_ylabel('Amplitude')
axes[0].set_title('Input ECG Signal')
axes[0].grid(True, alpha=0.3)

axes[1].plot(t[template_length//2:template_length//2+len(response)], 
             response, 'r', label='Filter Response')
axes[1].set_xlabel('Time (s)')
axes[1].set_ylabel('Response')
axes[1].set_title('Convolution Response (QRS Detection)')
axes[1].grid(True, alpha=0.3)

plt.tight_layout()
plt.show()

# Find peaks in response (detected heartbeats)
from scipy.signal import find_peaks
peaks, _ = find_peaks(response, height=0.5, distance=50)
print(f"Detected {len(peaks)} heartbeats")

_images/cec72d731f141dd2abc810f7f5cc45d62548fce8fd2ed4e19800ab62477484ba.png

Detected 9 heartbeats

10. Multiple Channels and Filters (Further Reading)#

Real-world applications often use multiple input channels and learn multiple features simultaneously.

10.1 Multi-Channel Input#

Consider a traffic monitoring system with multiple sensors:

Sensor	Measures	Unit	Relationship
Speed	How fast vehicles are moving	mph or km/h	Inverse relationship with congestion
Volume	How many vehicles pass a point	vehicles/hour	Increases with traffic
Occupancy	Percentage of road covered by vehicles	% (0-100)	Direct measure of congestion

Data shape: (3, 100)
Tensor shape: torch.Size([1, 3, 100])

traffic_data[:,:4], traffic_tensor[:,:,:4]

(array([[60.99342831, 60.72180556, 63.28207038, 66.00126178],
        [91.43300383, 96.81406504, 93.36332184, 86.98672098],
        [ 0.46432077,  0.49528602,  0.48847763,  0.45600965]]),
 tensor([[[60.9934, 60.7218, 63.2821, 66.0013],
          [91.4330, 96.8141, 93.3633, 86.9867],
          [ 0.4643,  0.4953,  0.4885,  0.4560]]]))

10.2 Multi-Output Convolution#

# Define layer: 3 input channels -> 5 output channels (filters)
conv_multi = nn.Conv1d(in_channels=3, out_channels=5, kernel_size=5, bias=True)

# Apply convolution
output = conv_multi(traffic_tensor)
print(f"Output shape: {output.shape}")
# Output: torch.Size([1, 5, 96])  (100 - 5 + 1 = 96)

# Each of the 5 output channels learns different patterns:
# - Channel 0: Might detect morning rush hour
# - Channel 1: Might detect accidents (sudden speed drop)
# - Channel 2: Might detect regular patterns
# etc.

Output shape: torch.Size([1, 5, 96])

10.3 Visualizing Learned Filters#

# Examine the learned weights
weights = conv_multi.weight.data  # Shape: (5, 3, 5)
print(f"Weights shape: {weights.shape}")

fig, axes = plt.subplots(5, 1, figsize=(10, 8))
for i in range(5):
    for j in range(3):
        axes[i].plot(weights[i, j].numpy(), 
                    label=f'Sensor {j}', marker='o')
    axes[i].set_title(f'Filter {i} weights')
    axes[i].legend()
    axes[i].grid(True, alpha=0.3)
plt.tight_layout()
plt.show()

11. Connection to 2D CNNs#

The concepts you’ve learned directly extend to 2D convolution for images:

Concept	1D (Time Series)	2D (Images)
Input shape	`(B, C, L)`	`(B, C, H, W)`
Kernel	`(out_ch, in_ch, k)`	`(out_ch, in_ch, k, k)`
Sliding direction	Along length	Along height and width
Detects	Temporal patterns	Spatial patterns (edges, textures)
Example	Heartbeat in ECG	Edge in image

The key insight is that convolution is pattern matching: in 1D we match temporal patterns, in 2D we match spatial patterns.

Summary#

Key Takeaways#

Convolution is local pattern detection: The kernel slides across the input, computing similarity at each position.
Kernel size controls receptive field: Larger kernels capture more context but reduce output size and may blur details.
Multiple channels enable rich representations: Each filter learns to detect different features.
1D CNNs are ideal for sequences: Time series, audio, and sensor data benefit from translation-invariant pattern detection.
Foundation for 2D: Understanding 1D convolution makes 2D convolution intuitive—just add another dimension.

Exercises#

Kernel Design: Design a kernel that detects sudden drops in a signal (anomaly detection). Test it on synthetic data.
Stride Experiment: Modify the smoothing example to use stride=2. How does this affect the output length and what does stride represent?
Padding Comparison: Compare padding=0 (valid) vs padding='same' in PyTorch. When would you use each?
Real Data: Download a time series dataset (e.g., from UCR Time Series Archive) and build a 1D CNN classifier.

This chapter provides a solid foundation before students encounter 2D convolutions for image processing. The progression from manual calculations → SciPy → PyTorch → real applications mirrors how students should internalize the concepts.

Introduction to 1D Convolutional Neural Networks

Contents

Introduction to 1D Convolutional Neural Networks#

Learning Objectives#

1. Why 1D Convolution?#

2. Mathematical Foundation of 1D Convolution#

2.1 The Convolution Operation#

2.2 Manual Example#

2.3 Different Kernel Effects#

3. Implementing 1D Convolution#

3.1 Using SciPy#

3.2 Using PyTorch#

4. Application: Signal Denoising#

4.1 Generating Noisy Data#

4.2 Effect of Kernel Size#

Understanding Phase Delay in Convolution#

The Phase Delay Phenomenon#

Visualizing the Effect#

4.3 The Trade-off: Smoothing vs Detail Preservation#

5. Pattern Detection with Template Matching#

5.1 The Concept: Convolution as Pattern Matching#

5.2 Example: Detecting a “Triangle” Pattern in Noise#

5.3 Applying Template Matching with Convolution#

5.4 Why This Works: The Mathematics#

5.5 Key Insights#

5.6 Real-World Connection: From 1D to 2D#

Exercise: Design Your Own Detector#

6. Multi-Output Convolution: Detecting Multiple Patterns#

6.1 The Setup: A Signal with Two Different Patterns#

6.2 Creating a Two-Channel Detector#

6.3 Applying and Visualizing Detection#

6.4 What This Means for Deep Learning#

Summary#

7. From Pattern Detection to Frequency Filtering#

7.1 Intuition: What’s “Slow” vs “Fast”?#

7.2 The Moving Average: A Simple Low-Pass Filter#

7.3 Why Kernel Size Matters#

7.4 High-Pass Filter: Keeping Only Fast Changes#

7.5 Connection to Pattern Detection#

8. Building a 1D CNN Classifier#

8.1 Architecture#

8.2 Training Loop Structure#

8.3 Application: Learn the Moving Average Kernel#

9. Application: ECG Heartbeat Detection (Further Reading)#

9.1 Understanding ECG Signals#

9.2 Simulating ECG Data#

9.3 Detecting R-peaks with Convolution#

10. Multiple Channels and Filters (Further Reading)#

10.1 Multi-Channel Input#

10.2 Multi-Output Convolution#

10.3 Visualizing Learned Filters#

11. Connection to 2D CNNs#

Summary#

Key Takeaways#

Further Reading#

Exercises#