Homework 4

Instructions

• Import Review.csv data first
• Run the provided code block (visible below) to prepare data and train a baseline model then answer the questions.
• Do not change the given R code.

---

Baseline Code (Run This First)

# Read Review.csv first.
# Review <- read.csv("data/Review.csv", header=FALSE)

# ============================================================
# Keras 3 (R: keras3) sentiment model for Korean reviews
# ============================================================

library(keras3)

set.seed(41)

review = Review
colnames(review) <- c("score", "comment")
review$score   <- as.integer(review$score)
review <- na.omit(review) # Remove NAs
review$score = ifelse(review$score <= 5, 0, 1) # Recode to 0-1 data

# --------------------------
# Train / valid / test split
# --------------------------
n   <- nrow(review) %/% 10
idx <- sample.int(n)
review <- review[idx, ]

n_test  <- max(1, round(0.20 * n))
test_set  <- review[seq_len(n_test), ]
train_set <- review[-seq_len(n_test), ]

x_train <- train_set$comment
x_test <- test_set$comment

y_train <- train_set$score
y_test  <- test_set$score

# ------------------------------------------------
# Text vectorization (minimal preprocessing)
# ------------------------------------------------
# - Keeps text as-is (no lowercasing/stripping) because it's Korean.
max_words <- 20000L
maxlen    <- 50L  # you can raise to reduce truncation

vec <- layer_text_vectorization(
  max_tokens = max_words,
  output_mode = "int",
  output_sequence_length = maxlen,
  standardize = NULL # keep Korean punctuation/spacing as-is
)

# Build the vocabulary from training text only
adapt(vec, train_set$comment)

# Save the whole vocabulary (not necessary)
vocab <- keras3::get_vocabulary(vec); head(vocab, 30)

# ------------------------------------------------
# Model: TextVectorization -> Embedding -> CNN
# ------------------------------------------------
embed_dim <- 128L

model <- keras3::keras_model_sequential(
  layers = list(
    vec,
    keras3::layer_embedding(input_dim = max_words, output_dim = embed_dim, mask_zero = FALSE),
    keras3::layer_spatial_dropout_1d(rate = 0.2),
    keras3::layer_conv_1d(filters = 128, kernel_size = 5, activation = "relu", padding = "same"),
    keras3::layer_conv_1d(filters = 128, kernel_size = 5, activation = "relu", padding = "same"),
    keras3::layer_global_max_pooling_1d(),
    keras3::layer_dropout(rate = 0.4),
    keras3::layer_dense(units = 64, activation = "relu"),
    keras3::layer_dropout(rate = 0.3),
    keras3::layer_dense(units = 1, activation = "sigmoid")
  )
)

model |> compile(
  optimizer = "adam",
  loss = "binary_crossentropy",
  metrics = "accuracy"
)

# ------------------------------------------------
# Train
# ------------------------------------------------
history <- model |>
  fit(
    x = x_train, y = y_train,
    epochs = 5,
    batch_size = 64,
    validation_split = 0.2,
    verbose = 2
  )

---

Questions

1) Print a readable model summary. How many parameters are there in total?

2) Print the accuracy using test dataset.

3) Compute predicted probabilities, convert to class labels with threshold 0.5, and report the confusion matrix.

4) Print the labels, predicted probabilities, and raw text for the three observations predicted to be the most interesting. Perform the same thing for the three observations predicted to be the least interesting.