This page shows visualizations of some width-1 1-d convolutional filters from Google's lm_1b language model. Each column corresponds to one position in the filter, and shows the characters with the most positive weights. Use the checkbox in the bottom-right to also see the most negative weights (may be slow).
Below that are examples of words for which the filter emits the highest values. A filter's response is its maximum value over all substrings it sees in the word. So if a filter has high weights on 'c' in the first position, then 'a', then 't', it will assign equally high scores to 'cat', 'fatcat', 'concatenate', etc. The portion of the string in blue is the substring the filter is responding to.
'^' and '$' represent beginning and end of word markers, respectively. '_' is a padding character. Literal versions of those characters are escaped with a backslash.
Use the links at the top to see filters of other widths.
Check out my blog post here for a bit more context.
Filter 0 (bias = -0.19) #
x
U
<BOS>
y
S
m
j
T
5
D
7
u
9
P
\194
L
V
z
4
H
6
d
Q
K
8
c
F
M
v
A
'
\195
X
r
N
O
0
t
b
G
Non-zero for 7.9% of words.
^nex t$____ (0.9) ^ex pected$____ (0.9) ^ex ecutive$____ (0.9) ^ex perience$____ (0.9) <BOS> ______(0.8) ^U.S .$____ (0.6) ^S he$____ (0.6) ^S tates$____ (0.6) ^S unday$____ (0.6) ^j ust$____ (0.4) ^maj or$____ (0.4) ^j ob$____ (0.4) ^j obs$____ (0.4) ^proj ect$____ (0.4) ^5 0$____ (0.4) ^5 00$____ (0.4) ^15 0$____ (0.4) ^5 5$____ (0.4) ^7 0$____ (0.4) ^7 5$____ (0.4) ^197 0s$____ (0.4) ^7 00$____ (0.4) ^7 2$____ (0.4) ^7 1$____ (0.4) ^\239\191 \189$____ (0.3) ^9 0$____ (0.3) ^19 99$____ (0.3) ^19 90s$____ (0.3) ^19 97$____ (0.3) ^\194 \163$____ (0.3) ^\194 \187$____ (0.3) ^\194 \189$____ (0.3) ^\194 \174$____ (0.3) ^V irginia$____ (0.3) ^V alley$____ (0.3) ^V egas$____ (0.3) ^V ice$____ (0.3) ^4 0$____ (0.2) ^4 5$____ (0.2) ^4 00$____ (0.2) Filter 1 (bias = -0.25) #
N
i
.
p
<BOS>
g
Q
m
F
k
/
P
K
Y
B
V
r
G
9
T
8
d
M
l
L
-
c
h
Z
1
o
H
f
v
A
b
R
z
E
J
Non-zero for 3.6% of words.
^N ew$____ (0.8) ^N orth$____ (0.8) ^N ational$____ (0.8) ^N ovember$____ (0.8) ^U. S.$____ (0.4) ^. ..$____ (0.4) ^p. m.$____ (0.4) ^a. m.$____ (0.4) <BOS> ______(0.3) ^Q ueen$____ (0.2) ^Q aeda$____ (0.2) ^Q uote$____ (0.2) ^al-Q aida$____ (0.2) ^al-Q aeda$____ (0.2) ^F riday$____ (0.2) ^F or$____ (0.2) ^F rench$____ (0.2) ^F rance$____ (0.2) ^K orea$____ (0.0) ^K ing$____ (0.0) ^K ennedy$____ (0.0) ^K orean$____ (0.0) ^K evin$____ (0.0) ^K ong$____ (0.0) Filter 2 (bias = -0.46) #
z
h
<BOS>
H
Q
y
v
F
U
M
I
j
\194
4
G
e
K
u
s
f
l
i
a
3
w
1
<EOS>
n
O
g
'
r
c
N
.
q
R
x
/
8
Non-zero for 9.8% of words.
^siz e$____ (2.1) ^organiz ation$____ (2.1) ^Gaz a$____ (2.1) ^magaz ine$____ (2.1) <BOS> ______(1.4) ^Q ueen$____ (0.3) ^Q aeda$____ (0.3) ^Q uote$____ (0.3) ^al-Q aida$____ (0.3) ^Ben\195\173 tez$____ (0.3) ^hav e$____ (0.2) ^ov er$____ (0.2) ^gov ernment$____ (0.2) ^ev en$____ (0.2) ^U .S.$____ (0.2) ^U nited$____ (0.2) ^U S$____ (0.2) ^U K$____ (0.2) ^I t$____ (0.2) ^I n$____ (0.2) ^I f$____ (0.2) ^I raq$____ (0.2) ^I ran$____ (0.2) ^I nternational$____ (0.2) ^\194 \163$____ (0.1) ^\194 \187$____ (0.1) ^\194 \189$____ (0.1) ^\194 \174$____ (0.1) Filter 3 (bias = -0.25) #
W
j
w
D
q
P
"
l
i
R
a
d
y
r
K
F
X
J
2
L
B
u
4
C
O
\195
^
-
,
0
G
s
g
z
U
Non-zero for 6.0% of words.
^W e$____ (1.1) ^W ednesday$____ (1.1) ^W ashington$____ (1.1) ^W hen$____ (1.1) ^w as$____ (0.6) ^w ith$____ (0.6) ^w ill$____ (0.6) ^w ho$____ (0.6) ^\226\130 \172$____ (0.2) ^\239\191 \189$____ (0.0) Filter 4 (bias = -0.40) #
.
i
L
P
A
p
Q
k
<BOS>
J
/
f
g
3
a
K
\194
B
l
1
q
,
m
9
N
R
Z
T
u
0
c
2
d
j
-
I
b
O
h
4
Non-zero for 4.3% of words.
^U. S.$____ (2.2) ^. ..$____ (2.2) ^p. m.$____ (2.2) ^U. N.$____ (2.2) ^L ondon$____ (0.4) ^L os$____ (0.4) ^L eague$____ (0.4) ^L ast$____ (0.4) ^A nd$____ (0.4) ^A merican$____ (0.4) ^A s$____ (0.4) ^A P$____ (0.4) ^A t$____ (0.4) ^A merica$____ (0.4) ^Q ueen$____ (0.2) ^Q aeda$____ (0.2) ^Q uote$____ (0.2) ^al-Q aida$____ (0.2) <BOS> ______(0.0) ^Ch\195\161 vez$____ (0.0) ^Guant\195\161 namo$____ (0.0) Filter 5 (bias = -0.08) #
1
m
W
f
q
L
I
F
g
M
R
l
k
K
v
.
Y
/
i
y
2
D
9
e
a
j
p
U
0
Z
7
t
"
s
h
P
3
<BOS>
-
S
Non-zero for 0.0% of words.
Filter 6 (bias = -0.25) #
C
v
G
q
P
e
s
b
1
E
D
x
z
f
R
w
n
W
/
B
c
-
A
.
5
N
7
u
,
h
g
<BOS>
I
a
V
M
Z
T
U
\163
Non-zero for 3.8% of words.
^C hina$____ (0.7) ^C ity$____ (0.7) ^C linton$____ (0.7) ^McC ain$____ (0.7) ^G eorge$____ (0.2) ^G eneral$____ (0.2) ^G overnment$____ (0.2) ^G roup$____ (0.2) Filter 7 (bias = -0.32) #
.
P
-
F
'
J
<BOS>
2
Y
B
x
i
m
3
Q
j
g
5
o
E
"
1
b
K
\194
4
h
0
q
6
r
I
a
D
/
e
l
9
u
T
Non-zero for 2.8% of words.
^U. S.$____ (1.1) ^. ..$____ (1.1) ^p. m.$____ (1.1) ^U. N.$____ (1.1) ^- -$____ (0.4) ^long- term$____ (0.4) ^e- mail$____ (0.4) ^so- called$____ (0.4) ^' s$____ (0.3) ^' t$____ (0.3) ^' re$____ (0.3) ^' ve$____ (0.3) ^' m$____ (0.3) ^' ll$____ (0.3) <BOS> ______(0.0) Filter 8 (bias = -0.34) #
.
i
F
p
Q
P
<BOS>
J
L
O
N
1
A
k
/
G
Z
o
\194
T
q
3
b
Y
c
0
'
K
n
2
X
R
x
I
D
z
"
-
a
\195
Non-zero for 6.5% of words.
^U. S.$____ (2.1) ^. ..$____ (2.1) ^p. m.$____ (2.1) ^U. N.$____ (2.1) ^F riday$____ (0.6) ^F or$____ (0.6) ^F rench$____ (0.6) ^F rance$____ (0.6) ^Q ueen$____ (0.5) ^Q aeda$____ (0.5) ^Q uote$____ (0.5) ^al-Q aida$____ (0.5) ^al-Q aeda$____ (0.5) ^Q atar$____ (0.5) <BOS> ______(0.5) ^L ondon$____ (0.5) ^L os$____ (0.5) ^L eague$____ (0.5) ^L ast$____ (0.5) ^N ew$____ (0.4) ^N orth$____ (0.4) ^N ational$____ (0.4) ^N ovember$____ (0.4) ^N o$____ (0.4) ^A nd$____ (0.4) ^A merican$____ (0.4) ^A s$____ (0.4) ^A P$____ (0.4) ^Z ealand$____ (0.2) ^Z imbabwe$____ (0.2) ^Z elaya$____ (0.2) ^Z ardari$____ (0.2) ^Z uma$____ (0.2) ^Z one$____ (0.2) ^\194 \163$____ (0.1) ^\194 \187$____ (0.1) ^\194 \189$____ (0.1) ^\194 \174$____ (0.1) Filter 9 (bias = -0.12) #
C
-
7
e
A
w
V
v
Q
E
1
o
,
J
H
f
s
b
4
<BOS>
R
m
U
q
S
.
F
T
n
M
8
N
k
\195
Y
K
Z
l
/
B
Non-zero for 2.6% of words.
^C hina$____ (0.3) ^C ity$____ (0.3) ^C linton$____ (0.3) ^McC ain$____ (0.3) Filter 10 (bias = -0.48) #
4
m
7
u
5
.
3
-
6
U
2
z
8
v
X
t
9
c
1
o
V
T
S
s
F
'
W
r
Q
d
0
w
H
b
,
l
I
y
i
k
Non-zero for 10.2% of words.
^4 0$____ (1.5) ^4 5$____ (1.5) ^4 00$____ (1.5) ^4 8$____ (1.5) ^7 0$____ (1.5) ^7 5$____ (1.5) ^197 0s$____ (1.5) ^7 00$____ (1.5) ^5 0$____ (1.4) ^5 00$____ (1.4) ^15 0$____ (1.4) ^5 5$____ (1.4) ^5 ,000$____ (1.4) ^25 0$____ (1.4) ^3 0$____ (1.3) ^3 1$____ (1.3) ^3 5$____ (1.3) ^3 00$____ (1.3) ^6 0$____ (1.2) ^6 5$____ (1.2) ^6 00$____ (1.2) ^196 0s$____ (1.2) ^6 -3$____ (1.2) ^6 -4$____ (1.2) ^2 008$____ (1.1) ^2 009$____ (1.1) ^2 0$____ (1.1) ^2 007$____ (1.1) ^8 0$____ (1.1) ^198 0s$____ (1.1) ^8 00$____ (1.1) ^8 5$____ (1.1) ^198 9$____ (1.1) ^198 8$____ (1.1) ^X inhua$____ (1.0) ^X VI$____ (1.0) ^X box$____ (1.0) ^X injiang$____ (1.0) ^9 0$____ (0.8) ^19 99$____ (0.8) Filter 11 (bias = -0.36) #
.
p
N
i
L
P
/
k
Q
T
\194
J
"
I
s
V
<BOS>
1
u
j
'
0
y
G
x
B
c
g
Z
\195
A
2
o
z
<EOS>
v
F
3
8
E
Non-zero for 3.1% of words.
^U. S.$____ (1.6) ^. ..$____ (1.6) ^p. m.$____ (1.6) ^U. N.$____ (1.6) ^N ew$____ (0.4) ^N orth$____ (0.4) ^N ational$____ (0.4) ^N ovember$____ (0.4) ^L ondon$____ (0.3) ^L os$____ (0.3) ^L eague$____ (0.3) ^L ast$____ (0.3) ^L abour$____ (0.3) ^L ee$____ (0.3) ^Q ueen$____ (0.1) ^Q aeda$____ (0.1) ^Q uote$____ (0.1) ^al-Q aida$____ (0.1) ^\194 \163$____ (0.1) ^\194 \187$____ (0.1) ^\194 \189$____ (0.1) ^\194 \174$____ (0.1) Filter 12 (bias = -0.11) #
Y
F
X
f
G
e
W
t
l
s
V
d
\194
c
H
j
o
E
<EOS>
n
J
p
^
P
L
I
T
C
\195
\163
b
x
m
S
z
,
/
k
"
r
Non-zero for 0.2% of words.
^Y ork$____ (0.3) ^Y ou$____ (0.3) ^Y ORK$____ (0.3) ^Y et$____ (0.3) Filter 13 (bias = -0.13) #
<BOS>
o
j
y
F
c
Q
m
I
K
V
U
7
w
X
O
4
u
6
z
\194
a
P
T
C
G
H
v
^
x
5
W
9
h
R
p
l
f
J
i
Non-zero for 4.0% of words.
<BOS> ______(0.3) ^j ust$____ (0.3) ^maj or$____ (0.3) ^j ob$____ (0.3) ^j obs$____ (0.3) ^F riday$____ (0.2) ^F or$____ (0.2) ^F rench$____ (0.2) ^F rance$____ (0.2) ^Q ueen$____ (0.2) ^Q aeda$____ (0.2) ^Q uote$____ (0.2) ^al-Q aida$____ (0.2) ^al-Q aeda$____ (0.2) ^I t$____ (0.2) ^I n$____ (0.2) ^I f$____ (0.2) ^I raq$____ (0.2) ^V irginia$____ (0.1) ^V alley$____ (0.1) ^V egas$____ (0.1) ^V ice$____ (0.1) ^V ietnam$____ (0.1) ^V an$____ (0.1) ^7 0$____ (0.1) ^7 5$____ (0.1) ^197 0s$____ (0.1) ^7 00$____ (0.1) ^X inhua$____ (0.1) ^X box$____ (0.1) Filter 14 (bias = -0.16) #
.
P
<BOS>
R
\194
p
L
i
q
k
l
U
A
O
X
r
t
G
v
3
e
J
m
K
/
s
h
\195
F
o
D
1
Q
9
j
f
M
y
N
C
Non-zero for 0.6% of words.
^U. S.$____ (0.2) ^. ..$____ (0.2) ^p. m.$____ (0.2) ^U. N.$____ (0.2) Filter 15 (bias = -0.34) #
H
x
I
v
A
z
1
c
N
p
Q
o
4
G
2
f
7
m
X
b
3
s
F
'
q
K
Z
-
/
g
r
0
u
S
t
<EOS>
^
k
j
.
Non-zero for 8.3% of words.
^H e$____ (1.8) ^H ouse$____ (1.8) ^H is$____ (1.8) ^H owever$____ (1.8) ^I t$____ (0.8) ^I n$____ (0.8) ^I f$____ (0.8) ^I raq$____ (0.8) ^A nd$____ (0.4) ^A merican$____ (0.4) ^A s$____ (0.4) ^A P$____ (0.4) ^A t$____ (0.4) ^A merica$____ (0.4) ^1 0$____ (0.2) ^1 2$____ (0.2) ^1 1$____ (0.2) ^1 5$____ (0.2) ^N ew$____ (0.2) ^N orth$____ (0.2) ^N ational$____ (0.2) ^N ovember$____ (0.2) ^N o$____ (0.2) ^N ews$____ (0.2) ^Q ueen$____ (0.2) ^Q aeda$____ (0.2) ^Q uote$____ (0.2) ^al-Q aida$____ (0.2) ^\226\128 \162$____ (0.2) ^\226\128 \154$____ (0.2) ^4 0$____ (0.1) ^4 5$____ (0.1) ^4 00$____ (0.1) ^4 8$____ (0.1) ^2 008$____ (0.1) ^2 009$____ (0.1) ^2 0$____ (0.1) ^2 007$____ (0.1) ^7 0$____ (0.1) ^7 5$____ (0.1) Filter 16 (bias = -0.48) #
j
y
<BOS>
U
7
m
J
a
0
K
R
w
\194
f
V
z
9
s
4
.
Y
p
I
L
6
A
X
c
5
x
D
W
g
o
S
,
H
O
-
B
Non-zero for 7.3% of words.
^j ust$____ (2.2) ^maj or$____ (2.2) ^j ob$____ (2.2) ^j obs$____ (2.2) <BOS> ______(0.9) ^7 0$____ (0.8) ^7 5$____ (0.8) ^197 0s$____ (0.8) ^7 00$____ (0.8) ^J ohn$____ (0.8) ^J une$____ (0.8) ^J uly$____ (0.8) ^J anuary$____ (0.8) ^J apan$____ (0.8) ^20 08$____ (0.6) ^20 09$____ (0.6) ^20 07$____ (0.6) ^20 06$____ (0.6) ^R euters$____ (0.5) ^R epublican$____ (0.5) ^R ussia$____ (0.5) ^R ussian$____ (0.5) ^R epublicans$____ (0.5) ^R obert$____ (0.5) ^\194 \163$____ (0.3) ^\194 \187$____ (0.3) ^\194 \189$____ (0.3) ^\194 \174$____ (0.3) ^V irginia$____ (0.3) ^V alley$____ (0.3) ^V egas$____ (0.3) ^V ice$____ (0.3) ^V ietnam$____ (0.3) ^V an$____ (0.3) ^9 0$____ (0.3) ^19 99$____ (0.3) ^19 97$____ (0.3) ^19 98$____ (0.3) ^4 0$____ (0.2) ^4 5$____ (0.2) Filter 17 (bias = -0.08) #
P
x
M
.
D
v
O
a
j
b
I
u
X
h
C
s
Q
-
T
z
F
w
H
m
3
q
1
l
r
o
K
B
2
<EOS>
7
A
Z
L
5
'
Non-zero for 0.0% of words.
Filter 18 (bias = -0.21) #
L
k
.
p
Z
t
/
i
8
T
D
I
5
v
6
r
A
R
l
P
F
'
X
C
G
q
N
-
K
O
m
W
<EOS>
\163
M
w
U
,
7
f
Non-zero for 2.0% of words.
^L ondon$____ (0.4) ^L os$____ (0.4) ^L eague$____ (0.4) ^L ast$____ (0.4) ^U. S.$____ (0.2) ^. ..$____ (0.2) ^p. m.$____ (0.2) ^U. N.$____ (0.2) ^Z ealand$____ (0.1) ^Z imbabwe$____ (0.1) ^Z elaya$____ (0.1) ^Z ardari$____ (0.1) ^Z uma$____ (0.1) ^Z one$____ (0.1) Filter 19 (bias = -0.31) #
Z
t
X
u
L
k
K
-
6
R
8
r
5
h
Q
s
2
v
G
T
/
j
V
o
W
i
7
'
3
f
4
S
^
d
9
g
m
I
<EOS>
C
Non-zero for 7.5% of words.
^Z ealand$____ (1.7) ^Z imbabwe$____ (1.7) ^Z elaya$____ (1.7) ^Z ardari$____ (1.7) ^X inhua$____ (1.7) ^X VI$____ (1.7) ^X box$____ (1.7) ^X injiang$____ (1.7) ^L ondon$____ (1.2) ^L os$____ (1.2) ^L eague$____ (1.2) ^L ast$____ (1.2) ^L abour$____ (1.2) ^L ee$____ (1.2) ^K orea$____ (1.2) ^K ing$____ (1.2) ^K ennedy$____ (1.2) ^K orean$____ (1.2) ^6 0$____ (0.7) ^6 5$____ (0.7) ^6 00$____ (0.7) ^196 0s$____ (0.7) ^6 -3$____ (0.7) ^6 -4$____ (0.7) ^8 0$____ (0.5) ^198 0s$____ (0.5) ^8 00$____ (0.5) ^8 5$____ (0.5) ^5 0$____ (0.5) ^5 00$____ (0.5) ^15 0$____ (0.5) ^5 5$____ (0.5) ^5 ,000$____ (0.5) ^25 0$____ (0.5) ^Q ueen$____ (0.5) ^Q aeda$____ (0.5) ^Q uote$____ (0.5) ^al-Q aida$____ (0.5) ^2 008$____ (0.4) ^2 009$____ (0.4) Filter 20 (bias = -0.33) #
N
k
E
v
O
p
L
m
K
g
Q
'
U
V
2
C
3
i
8
n
/
x
"
c
9
j
A
b
e
-
I
h
a
t
W
l
r
T
o
\194
Non-zero for 7.1% of words.
^N ew$____ (1.1) ^N orth$____ (1.1) ^N ational$____ (1.1) ^N ovember$____ (1.1) ^E ngland$____ (1.1) ^E uropean$____ (1.1) ^E urope$____ (1.1) ^E ast$____ (1.1) ^O bama$____ (0.6) ^O n$____ (0.6) ^O ne$____ (0.6) ^O ctober$____ (0.6) ^O ffice$____ (0.6) ^O f$____ (0.6) ^L ondon$____ (0.6) ^L os$____ (0.6) ^L eague$____ (0.6) ^L ast$____ (0.6) ^K orea$____ (0.5) ^K ing$____ (0.5) ^K ennedy$____ (0.5) ^K orean$____ (0.5) ^K evin$____ (0.5) ^K ong$____ (0.5) ^Q ueen$____ (0.4) ^Q aeda$____ (0.4) ^Q uote$____ (0.4) ^al-Q aida$____ (0.4) ^U .S.$____ (0.4) ^U nited$____ (0.4) ^U S$____ (0.4) ^U niversity$____ (0.4) ^U K$____ (0.4) ^U PI$____ (0.4) ^2 008$____ (0.4) ^2 009$____ (0.4) ^2 0$____ (0.4) ^2 007$____ (0.4) ^3 0$____ (0.4) ^3 1$____ (0.4) Filter 21 (bias = -0.45) #
.
i
A
J
Q
K
c
P
g
3
'
p
h
O
b
o
C
f
\194
2
l
1
t
E
L
9
F
W
a
M
q
6
<BOS>
4
/
5
d
w
r
,
Non-zero for 2.9% of words.
^U. S.$____ (0.8) ^. ..$____ (0.8) ^p. m.$____ (0.8) ^U. N.$____ (0.8) ^A nd$____ (0.2) ^A merican$____ (0.2) ^A s$____ (0.2) ^A P$____ (0.2) Filter 22 (bias = -0.41) #
.
s
q
U
Q
G
<BOS>
J
\194
i
X
K
v
P
r
S
a
E
b
3
N
z
'
O
t
5
n
L
"
o
h
f
W
R
H
0
\163
M
-
4
Non-zero for 1.6% of words.
^U. S.$____ (1.2) ^. ..$____ (1.2) ^p. m.$____ (1.2) ^U. N.$____ (1.2) ^q uarter$____ (1.0) ^q uestion$____ (1.0) ^q uestions$____ (1.0) ^q uickly$____ (1.0) ^Q ueen$____ (0.4) ^Q aeda$____ (0.4) ^Q uote$____ (0.4) ^al-Q aida$____ (0.4) ^al-Q aeda$____ (0.4) ^Q atar$____ (0.4) <BOS> ______(0.3) ^\239 \191\189$____ (0.2) ^\194 \163$____ (0.1) ^\194 \187$____ (0.1) ^\194 \189$____ (0.1) ^\194 \174$____ (0.1) ^X inhua$____ (0.0) ^X VI$____ (0.0) ^X box$____ (0.0) ^X injiang$____ (0.0) Filter 23 (bias = -0.09) #
<BOS>
u
x
U
.
T
X
k
Q
H
5
i
7
h
\194
R
6
y
8
t
G
B
l
r
^
m
2
M
p
Y
0
P
/
A
9
\195
V
1
Z
D
Non-zero for 3.7% of words.
<BOS> ______(0.4) ^nex t$____ (0.3) ^ex pected$____ (0.3) ^ex ecutive$____ (0.3) ^ex perience$____ (0.3) ^U. S.$____ (0.3) ^. ..$____ (0.3) ^p. m.$____ (0.3) ^U. N.$____ (0.3) ^X inhua$____ (0.2) ^X VI$____ (0.2) ^X box$____ (0.2) ^X injiang$____ (0.2) ^X avier$____ (0.2) ^Q ueen$____ (0.2) ^Q aeda$____ (0.2) ^Q uote$____ (0.2) ^al-Q aida$____ (0.2) ^5 0$____ (0.2) ^5 00$____ (0.2) ^15 0$____ (0.2) ^5 5$____ (0.2) ^5 ,000$____ (0.2) ^25 0$____ (0.2) ^7 0$____ (0.1) ^7 5$____ (0.1) ^197 0s$____ (0.1) ^7 00$____ (0.1) ^\194 \163$____ (0.1) ^\194 \187$____ (0.1) ^\194 \189$____ (0.1) ^\194 \174$____ (0.1) ^6 0$____ (0.1) ^6 5$____ (0.1) ^6 00$____ (0.1) ^196 0s$____ (0.1) ^\239\191 \189$____ (0.1) ^8 0$____ (0.0) ^198 0s$____ (0.0) ^8 00$____ (0.0) Filter 24 (bias = -0.24) #
F
-
S
o
Q
u
5
\195
C
J
8
w
4
v
7
T
6
r
,
Y
X
b
/
R
L
z
\194
i
f
k
D
m
s
g
e
q
j
p
2
a
Non-zero for 8.5% of words.
^F riday$____ (1.1) ^F or$____ (1.1) ^F rench$____ (1.1) ^F rance$____ (1.1) ^U.S .$____ (0.4) ^S he$____ (0.4) ^S tates$____ (0.4) ^S unday$____ (0.4) ^Q ueen$____ (0.3) ^Q aeda$____ (0.3) ^Q uote$____ (0.3) ^al-Q aida$____ (0.3) ^al-Q aeda$____ (0.3) ^Q atar$____ (0.3) ^5 0$____ (0.3) ^5 00$____ (0.3) ^15 0$____ (0.3) ^5 5$____ (0.3) ^C hina$____ (0.3) ^C ity$____ (0.3) ^C linton$____ (0.3) ^McC ain$____ (0.3) ^C ourt$____ (0.3) ^C alifornia$____ (0.3) ^8 0$____ (0.2) ^198 0s$____ (0.2) ^8 00$____ (0.2) ^8 5$____ (0.2) ^4 0$____ (0.1) ^4 5$____ (0.1) ^4 00$____ (0.1) ^4 8$____ (0.1) ^4 2$____ (0.1) ^4 4$____ (0.1) ^7 0$____ (0.1) ^7 5$____ (0.1) ^197 0s$____ (0.1) ^7 00$____ (0.1) ^6 0$____ (0.1) ^6 5$____ (0.1) Filter 25 (bias = -0.21) #
W
u
x
-
X
D
B
d
V
r
S
R
K
j
<BOS>
U
\194
J
Q
P
5
\195
"
g
6
s
,
I
b
E
w
T
p
L
Y
H
4
1
q
o
Non-zero for 9.0% of words.
^W e$____ (0.9) ^W ednesday$____ (0.9) ^W ashington$____ (0.9) ^W hen$____ (0.9) ^nex t$____ (0.6) ^ex pected$____ (0.6) ^ex ecutive$____ (0.6) ^ex perience$____ (0.6) ^X inhua$____ (0.3) ^X VI$____ (0.3) ^X box$____ (0.3) ^X injiang$____ (0.3) ^X avier$____ (0.3) ^FOX News.com$____ (0.3) ^B ut$____ (0.3) ^B ritish$____ (0.3) ^B ush$____ (0.3) ^B ritain$____ (0.3) ^V irginia$____ (0.3) ^V alley$____ (0.3) ^V egas$____ (0.3) ^V ice$____ (0.3) ^V ietnam$____ (0.3) ^V an$____ (0.3) ^U.S .$____ (0.2) ^S he$____ (0.2) ^S tates$____ (0.2) ^S unday$____ (0.2) ^K orea$____ (0.2) ^K ing$____ (0.2) ^K ennedy$____ (0.2) ^K orean$____ (0.2) ^K evin$____ (0.2) ^K ong$____ (0.2) <BOS> ______(0.2) ^\194 \163$____ (0.1) ^\194 \187$____ (0.1) ^\194 \189$____ (0.1) ^\194 \174$____ (0.1) ^\239\191 \189$____ (0.1) Filter 26 (bias = -0.36) #
B
m
N
d
E
c
<BOS>
p
9
y
J
g
R
.
S
-
3
D
4
'
I
G
F
C
2
z
5
v
K
l
W
n
,
\163
7
L
8
x
6
T
Non-zero for 14.9% of words.
^B ut$____ (1.2) ^B ritish$____ (1.2) ^B ush$____ (1.2) ^B ritain$____ (1.2) ^N ew$____ (0.9) ^N orth$____ (0.9) ^N ational$____ (0.9) ^N ovember$____ (0.9) ^E ngland$____ (0.9) ^E uropean$____ (0.9) ^E urope$____ (0.9) ^E ast$____ (0.9) ^E ven$____ (0.9) ^E U$____ (0.9) <BOS> ______(0.6) ^9 0$____ (0.6) ^19 99$____ (0.6) ^19 90s$____ (0.6) ^19 97$____ (0.6) ^J ohn$____ (0.6) ^J une$____ (0.6) ^J uly$____ (0.6) ^J anuary$____ (0.6) ^J apan$____ (0.6) ^R euters$____ (0.6) ^R epublican$____ (0.6) ^R ussia$____ (0.6) ^R ussian$____ (0.6) ^U.S .$____ (0.6) ^S he$____ (0.6) ^S tates$____ (0.6) ^S unday$____ (0.6) ^S aturday$____ (0.6) ^S outh$____ (0.6) ^3 0$____ (0.6) ^3 1$____ (0.6) ^3 5$____ (0.6) ^3 00$____ (0.6) ^4 0$____ (0.4) ^4 5$____ (0.4) Filter 27 (bias = -0.17) #
I
x
<BOS>
h
Q
v
/
b
,
m
2
c
P
g
N
k
R
y
O
f
n
B
D
G
H
Y
t
V
A
p
^
M
C
T
1
o
7
S
X
0
Non-zero for 1.1% of words.
^I t$____ (0.6) ^I n$____ (0.6) ^I f$____ (0.6) ^I raq$____ (0.6) <BOS> ______(0.2) ^Q ueen$____ (0.2) ^Q aeda$____ (0.2) ^Q uote$____ (0.2) ^al-Q aida$____ (0.2) ^\226\128 \162$____ (0.0) Filter 28 (bias = -0.38) #
A
-
S
v
Q
u
<BOS>
d
N
m
E
c
O
D
W
P
I
n
X
'
F
U
K
T
,
J
5
C
B
y
r
p
e
z
t
M
4
i
a
0
Non-zero for 10.6% of words.
^A nd$____ (1.3) ^A merican$____ (1.3) ^A s$____ (1.3) ^A P$____ (1.3) ^U.S .$____ (1.2) ^S he$____ (1.2) ^S tates$____ (1.2) ^S unday$____ (1.2) ^Q ueen$____ (0.9) ^Q aeda$____ (0.9) ^Q uote$____ (0.9) ^al-Q aida$____ (0.9) ^al-Q aeda$____ (0.9) ^Q atar$____ (0.9) <BOS> ______(0.8) ^N ew$____ (0.8) ^N orth$____ (0.8) ^N ational$____ (0.8) ^N ovember$____ (0.8) ^E ngland$____ (0.6) ^E uropean$____ (0.6) ^E urope$____ (0.6) ^E ast$____ (0.6) ^E ven$____ (0.6) ^O bama$____ (0.4) ^O n$____ (0.4) ^O ne$____ (0.4) ^O ctober$____ (0.4) ^W e$____ (0.3) ^W ednesday$____ (0.3) ^W ashington$____ (0.3) ^W hen$____ (0.3) ^W orld$____ (0.3) ^W est$____ (0.3) ^I t$____ (0.2) ^I n$____ (0.2) ^I f$____ (0.2) ^I raq$____ (0.2) ^X inhua$____ (0.1) ^X VI$____ (0.1) Filter 29 (bias = -0.33) #
k
L
T
l
'
D
U
o
r
.
W
5
B
J
"
d
i
/
V
6
R
-
t
j
C
0
b
G
p
7
v
N
I
K
Q
X
\163
<EOS>
H
n
Non-zero for 4.3% of words.
^lik e$____ (0.9) ^mak e$____ (0.9) ^tak e$____ (0.9) ^mark et$____ (0.9) Filter 30 (bias = -0.51) #
.
P
-
J
'
i
\194
K
<BOS>
k
n
B
m
3
l
p
d
U
t
E
q
R
g
2
/
9
h
1
x
G
c
T
A
0
Q
O
L
\195
v
4
Non-zero for 2.9% of words.
^U. S.$____ (2.2) ^. ..$____ (2.2) ^p. m.$____ (2.2) ^U. N.$____ (2.2) ^- -$____ (0.5) ^long- term$____ (0.5) ^e- mail$____ (0.5) ^so- called$____ (0.5) ^' s$____ (0.2) ^' t$____ (0.2) ^' re$____ (0.2) ^' ve$____ (0.2) ^' m$____ (0.2) ^' ll$____ (0.2) ^\194 \163$____ (0.1) ^\194 \187$____ (0.1) ^\194 \189$____ (0.1) ^\194 \174$____ (0.1) <BOS> ______(0.0) ^\239\191 \189$____ (0.0) Filter 31 (bias = -0.33) #
O
n
W
u
S
v
E
F
G
k
Y
C
"
j
Q
f
o
R
i
J
w
Z
p
-
y
b
A
M
2
P
K
B
a
D
,
m
^
'
t
9
Non-zero for 6.9% of words.
^O bama$____ (1.6) ^O n$____ (1.6) ^O ne$____ (1.6) ^O ctober$____ (1.6) ^W e$____ (1.3) ^W ednesday$____ (1.3) ^W ashington$____ (1.3) ^W hen$____ (1.3) ^U.S .$____ (1.0) ^S he$____ (1.0) ^S tates$____ (1.0) ^S unday$____ (1.0) ^S aturday$____ (1.0) ^S outh$____ (1.0) ^E ngland$____ (0.8) ^E uropean$____ (0.8) ^E urope$____ (0.8) ^E ast$____ (0.8) ^\226\130 \172$____ (0.5) ^G eorge$____ (0.3) ^G eneral$____ (0.3) ^G overnment$____ (0.3) ^G roup$____ (0.3) ^G ermany$____ (0.3) ^Y ork$____ (0.1) ^Y ou$____ (0.1) ^Y et$____ (0.1) ^Y oung$____ (0.1) ^Q ueen$____ (0.1) ^Q aeda$____ (0.1) ^Q uote$____ (0.1) ^al-Q aida$____ (0.1) ^al-Q aeda$____ (0.1) ^Q atar$____ (0.1)